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ABSTRACT 



A method and apparatus for high speed conversion of 
tangible source documents to electronic images, and subse- 
quent transmission or storage and retrieval of images, uti- 
lizing hybrid signal processing. Hie system employs a 
higher bandwidth analog signal for image capture and lower 
effective bandwidth analog signal for transmission or stor- 
age and retrieval, with an intervening digital memory uti- 
lized to construct a bitmap of the image to facilitate various 
dissection and seaming schemes which optimize image 
content and processing time. The system is designed around 
a conventional bus stmcture, anid the memory serves as a 
junction with conventional personal computers, networks, 
and peripheral devices. In a representative embodiment, a 
tangible image is captured using a camera producing an 
analog signal with conventional raster synchronization. The 
synchronization is stripped from the analog signal, which is 
digitized for 8-bit grayscale and multiplexed to the memory 
where the image exists as a bitmap that may be divided into 
segments. The content is read from the memory, converted 
to an analog signal, and control signals are added. The 
control signals include horizontal and vertical sync pulses 
and interval blanking, a pilot signal to maintain alignment 
^ between adjacent segments along seams and to compensate 
for time-based errors, and calibration pulses to permit 
instantaneous adjustment of the gray level for each line, 
ensure accurate image content, and permit display enhance- 
ment. The resultant analog signal is stored on a randomly 
accessible storage medium as one or more frames, transmit- 
ted and reassembled, or displayed on a conventional moni- 
tor 

51 Claims, 6 Drawing Sheets 
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METHOD AND APPARATUS FOR THE 

TRANSMISSION, STORAGE, AND 
RETRIEVAL OF DOCUMENTS IN AN 
ELECTRONIC DOMAIN 

5 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

This invention relates generally to interactive document 
processing in an electronic domain, and particularly to a new i q 
modality incorporating hybrid (digital and analog) signal 
processing for the transmission, storage, and retrieval of 
documents to optimize informational content and processing 
time. 

2. Genera] Terminology is 
While the terminology utilized in the detailed description 

of the preferred embodiments is basic to those of ordinary 
skill in the art relating to designing equipment of the types 
discussed herein (i.e., television cameras, optical laser disc 
recorders, personal computer interfaces, local area networks, ^ 
data transmission systems), the general terminology relating 
to interactive document processing operations is uncertain. 
A clear and uniform delineation of this terminology is 
therefore considered necessary to a proper understanding of 
the subject matter of this invention, ^ 

A document is any information recorded in a substantial 
(tangible) or insubstantial (electronic) form structured for 
human visual comprehension either directly or through the 
aid of enabling equipment. Documents were at one time 
considered written or tangible information, and this defini- 
tion was later broadened to information fixed in any tangible 
medium such as magnetic tapes or optical disks. Currently, 
it is necessary to view documents even more broadly, since 
they may be created and reside exclusively in electronic 
memory, and may be transformed, presented, or transmitted 
without ever being reduced to a tangible medium. 

Documents are therefore one subset of all the types of 
informational bundles, with non-documents generally being 
refened to simply as recordings. Documents and recordings ^ 
may coexist in the same tangible medium, such as a mag- 
netic video tape having both an audio recording coupled 
with a sequence of visual frames, or they may exist in related 
or completely separate and distinct mediums. 

The differentiating requirement imposed upon documents 45 
is that they be capable of direct or assisted visual compre- 
hension. For example, a page written in braille is normally 
structured for both visual and tactile comprehension. The 
fact that it is intended for tactile comprehension does not 
defeat the fact that it is also structured for visual compre- 50 
hension. (In this instance, comprehension does not imply 
recognition or interpretation of the braille characters or their 
assigned meanings, but merely acknowledgement or appre- 
ciation of their existence as having content and structured 
form in the document which may be visually observed.) 55 
Several types of informational bundles that are structm:ed for 
comprehension only by senses other than visual or auditory 
do exist or have been postulated, but no uniform or concise 
body of terminology has been developed to classify or 
categorize their nature or properties. Furthermore, only go 
recordings and documents have developed as recognized 
means for information processing and interpersonal com- 
munication. 

It may be appreciated that where documents are integrated 
with recordings, the term "recordings" has been adopted as 65 
the descriptive or identifying name for both components of 
the informational bundle predominantly because of the 
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particular nature of the existing technologies and mediums 
currently employed on a standardized basis. It remains to be 
seen whether this tendency will change with the prolifera- 
tion of multimedia processing (that is, processing both 
documents and recordings in an electronic domain as inte- 
grated but separable components of an unitary informational 
bundle.) 

Although documents existed and were used for interper- 
sonal communication long before recordings, the techno- 
logical improvements in recording over the last century have 
so overshadowed the improvements in processing tangible 
documents that the basic recording technologies were 
applied extensively and uniformly to both tangible and 
electronic docimients. The cunent methods utilized for 
processing both substantial and insubstantial documents 
have therefore remained conceptually static since the intro- 
duction of electronic documents, due in large part to the 
limiting effect caused by the recent revolution in semicon- 
ductor memory and the correspondingly complete accep- 
tance of a digital standard controlling both quantitative and 
qualitative precision for all insubstantial documents. 

A document may exist within any one of three domains: 
tangible (paper, microfilm or fiche, photographic negatives 
or prints, etc.); electronic image (two dimensional bitmaps 
or arrays consisting of rows and columns of pixels each 
having a particular informational depth describing a mono- 
chromatic, grayscale, or color value); and electronic content 
(alphanumeric data strings, formatted text, spreadsheets, 
machine-readable programs, etc) 

There have traditionally been considered six basic opera- 
tions that may be performed in association with a document: 
creation, reproduction, transformation, display, communica- 
tion, and storage and retrieval. These operations may be 
performed on any document regardless of the domain in 
which it resides. The commonly accepted definitions for 
each these operations were distilled and articulated concur- 
rently with the development of the existing technologies for 
converting tangible documents to electronic images, and for 
converting portions of electronic images, to electronic con- 
tent. 

As may be seen by the following discussion, recent 
developments in the field of document processing (and 
particularly the technology disclosed in this application) 
necessitate expanding this list and refining the definitions of 
those operations. An appreciation of the interrelationships 
among and distinctions between these operations and their 
intended definitions is tiiierefore a prerequisite to understand- 
ing the modality disclosed herein. 

The definitions of ''creation" and "reproduction" as used 
herein differ only slightly from the traditional protocol. 

Creation is the initial authoring and fixing of a document 
in a specific medium. Creation can then be said to comprise 
the interrelated steps of composition and recordation in 
which the document is given content and form, both of 
which may be dependent on human perception and physical 
limitations of the medium. 

Reproduction is the recording of a document's current 
content and form on multiple instances of the document's 
current medium. 

At this point it is necessary to diverge more significantly 
from the existing protocol to interpose distinct and broader 
operational terms. As discussed subsequently in greater 
detail, current technology has produced expectations regard- 
ing operational precision in document processing that have 
focussed principally on the qualitative precision in repro- 
duction of tangible documents and electronic images and the 
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quantitative precision in transmission, storage, and retrieval 
of electronic content. These expectations are no longer valid, 
particularly when discussing the qualitative precision asso- 
ciated with document processing in the electronic image 
domain, and to be accurate the terms must therefore reflect 
the fact that certain processes affect either the content or 
form of a document to the degree that those processes must 
be reclassified as different operations. 

For example, photocopying has traxiitionally been thought 
of as reproduction since it is the production of a "duplicate" 
of an original image on a similar tangible medium. This 
duplicate image retains sufficient qualitative precision in 
both content and form that it may be utilized for interper- 
sonal communication in place of the original for many legal 
or business practices. However, current black-and-white and 
color photocopying processes result in the loss of such a 
substantial amount of information in some situations that it 
constitutes a transformation or material alteration in the 
basic content of the original document compared with the 
levels of qualitative precision established by the technology 
disclosed herein. 

As such, for purposes of this discussion photocopying is 
actually the creation of a distinct derivative document based 
upon the form and content of the original. In some instances, 
due to the nature of the original document and the photo- 
copying technique employed, the derivative document will 
retain sufficient information and qualitative precision to 
constitute a reproduction. However, a photocopy will not in 
all cases be a reproduction. 

Representation is the recording of a document's current 
content and form on a different or distinct medium, or 
recording a portion of the current content and fonn on the 
same or different medium, which results in a change in the 
informational composition of the document. Representation 
may therefore be thought of as involving some intermediate 
transition in the domain, content, or form of a document. 
Representation will frequently entail the transition between 
two domains, such as the printing of an electronic image on 
a tangible medium or rendering the image as electronic 
content, but as with reproduction the original document may 
continue to exist and reside in the same domain. 

Because documents are more frequently being created in 
the electronic image and electronic content domains, it is 
important to remember that many people incorrectly regard 
the initial tangible representation of a document as the 
original. Those representations aie in fact only derivative 
documents which do not possess the same informational 
composition as the original, and many representations of an 
original document result in transitions or transformations 
that are unintentional from the viewpoint of the operator, but 
which are intentional and necessary from the viewpoint of 
the designer of the technology being utilized to produce the 
representation. 

Thus, any intermediate transition between domains or 
mediums is assumed to encompass some form of a repre- 
sentation, unless the particular nature and character of the 
original document and the processes used for the transition 
are sufficiently precise and compatible for the representation 
to be considered a reproduction for the given functional 
purposes being considered. 

IVansformation is the change in content, form, medium, 
or domain of a document that produces a new or derivative . 
version which is itself a unique document. Transformation 
can easily be thought of as the creation of a new document, 
and conversely the process of creation can be thought of as 
a series of transformations eventually resulting in a docu- 
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mcnt having a desired informational content or form within 
a specified medium. In some cases (such as manipulating an 
electronic image contained in semiconductor memory) the 
original document ceases to exist and is replaced by the new 
or derivative document It should also be remembered that 
other operations such as representation may inherently pro- 
duce or require transformations due either to the technology 
enqiloyed or the limitations imposed by transitions between 
mediums or domains, and those transformations and 
manipulations are frequently transparent to or not appreci- 
ated by the operator. As with representation, transformation 
involves some intendonal transition in the document 
invoked by the operator or the technology designer. The 
term manipulation is therefore regarded as being more 
appropriate to describe transformations that are conscioiisly 
made, selected, instructed, or invoked directiy by the opera- 
tor to intentionally affect the content or form of the docu- 
ment in a predetermined manner. 

Transformation has traditionally included some processes 
involving a change in medium or domain, however since we 
must assume for definitional purposes that a transition to a 
new medium or domain may have a substantial and often 
deleterious affect on the actual informational content of a 
document, any process involving the transition between 
mediums or domains are also necessarily regarded as a 
transformation that gives rise to either a representation of the 
document or the creation of a new document. 

Presentation is the visual manifestation of a selected 
portion of the content and form of a document for human 
comprehension. Presentation would include processes such 
as displaying an electronic image as a rasterized image on a 
cathode-ray tube (CRT), as a bitmap image on a liquid 
crystal display (LCD) or Ught-emitting diode (LED) display, 
or the process of projecting a visible image onto a tangible 
surface using an LCD, LED, or similar device. Presentation 
would also include other methods of projection, such as 
re&actively projecting a visible image from a tangible 
document such as film, fiche, slides, negatives, or X-rays. 

Transmission is the transportation of a document through 
space or between remote locations. Transmission is believed 
to be a more accinrate term than communication, since a 
document may be satisfactorily communicated and compre- 
hended without requiring transmission (such as by display- 
ing or representing the document.) 

One could theoretically distinguish communication by 
defining cognitive boundaries for each individual involved 
in the processes of document handling (such as the author/ 
creator, editor/operator, interpreter/reader) and treat commu- 
nication as transporting the informational content of the 
document between cognitive boundaries. In comparison, 
transmission is the physical transportation of a document 
through space without regard or reference to cognitive 
boundaries. Transmission may then be thought of as a subset 
of communication, but devoid of any comprehension 
requirements. To the extent that communication requires 
comprehension of information at some level, it is strictiy 
speddng not an operation that is performed on a document, 
but simply the utilization of one or more document process- 
ing operations as intermediate steps in the overall process to 
achieve the result of interpersonal communicatioa 

Storage and Retrieval is the transportation of a document 
through time in a static state in a manner permitting tiie 
selective acquisition of that individual document from its 
storage medium. The provision that the document be in a 
static state is an addition to the traditional definition of 
storage and retrieval, since a document may theoretically be 
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transported through time by holding the document in active 
memory without being stored and retrieved. Similarly, the 
provision that the document be selectively acquired firom its 
storage medium is an addition to the traditional definition, 
distinguishing storage and retrieval from the independent 
and unrelated operations of recording and replaying. 

An electronic image normally exists in active volatile 
memory, and what is perceived by the operator as the 
document is actually a display of the document being 
repeatedly and instantaneously refreshed by information 
drawn from that memory, and manipulations of the dis- 
played image are inseiteii into and held in that memory. 
Alternately, an electronic image may be swapped to a 
separate portion of memory such as a volatile RAM disk or 
cache, which simulates the existence of a magnetic or optical 
storage medium but at higher speeds. The process of 
exchanging informational bits in active memory, and the 
intervening holding of those informational bits, is not con- 
sidered storage and retrieval. Conversely, one could consider 
the process of writing an electronic image to nonvolatile 
semiconductor memory such as read-only memory (ROM) 
to accomplish the storage and retrieval function, however 
semiconductor memory is rarely (if ever) utilized for the 
selective acquisition of electronic images. 

The fact that storage and retrieval have conventionally 
been treated within the boundaries of one operation under- 
scores the unique and reciprocal nature of those two pro- 
cesses, wherein storage implies the ability to retrieve 
selected documents individually and non-sequentially from 
among a plurality of sequentially indexed documents. TTie 
nature of document storage may be contrasted with that for 
audio recordings, where there is no reciprocal function 
corresponding to retrieval. Instead, recordings are either 
reproduced or they are replayed in a marmer corresponding 
to presentation or representation. Thus, for most purposes 
the "storage and retrieval" function for recordings forms a 
closed loop through various other operations, and is nor- 
mally not treated as a reciprocal function or single operation. 
Electronic images may be processed through very similar 
dosed loops involving other operations than storage and 
retrieval when those docimients are being treated as (or form 
portions of) recordings, such as in the case of multimedia 
processing applications, but these closed loops are not 
deemed to be storage and retrieval. 

The execution of one or more of the operations described 
above is termed "document processing,*' which is consid- 
ered to be a subset of **infonnation processing" since infor- 
mation can exist in forms other than documents. Hiere are 
similar sets of operations applied to processing information 
in fields other than the three document domains, each 
operation having a definition which may be unique or 
peculiar to that field. However, there is frequently some 
overlap between the terminology used in document process- 
ing and other fields of information processing, as well as the 
informal or nontechnical use of the same terminology, which 
can result in some inaccuracy and inconsistency. 

For example, electronic content is frequentiy but errone- 
ously referred to as "data" because people equate that term 
for computer fties with the discrete elements of visual 
images that they recognize as conveying exact or immutable 60 
information. A person recognizes certain elements (such as 
numbers, letters, or symbols we term "characters") with a 
degree of precision that corresponds to their appreciation of 
how computers read data, without reference to the concep- 
tual principles involved with communicating, xmderstand- 65 
ing, or comprehending the content of that information apart 
from the incremental elements. A person can view a page of 
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text and differentiate discrete elements such as characters, 
each of which have an assigned meaning to that individual. 
The informational content of the page is considered rela- 
tively precise or exact, since it is presimied that individuals 
to whom the document is directed will assign the same 
meanings to each element. For purposes of determining' 
precision or accuracy, we disregard both the fact that the 
aggregation and interpretation of those discrete elements 
may convey completely different concepts to each indi- 
vidual, and we dispense with individuals who assign differ- 
ent meanings to those informational elements than do mem- 
bers of the target group. In the same way, people understand 
computers to read data with exact accuracy independent 
from the need for recognition or interpretation, and there are 
basic hi^archical groupings of formats, programs, lan- 
guages, and machine architectures that define what mean- 
ings will be assigned to discrete data elements for certain 
purposes. This appreciation may change as the general 
public becomes more aware of and familiar with the tech- 
nological processes involved in the conversion of electronic 
images to electronic content through optical character rec- 
ognition (OCR), the application of artificial intelligence or 
fuzzy logic to visual object recognition in robotics systems, 
or the use of electronic photography to store electronic 
images on optical recording medium in place of film nega- 
tives. 

Electronic content can more properly be defined as 
encoded data plus a set of structural linkages. The data or 
information is encoded so that it may be utilized or operated 
on directiy without human or artificial intelligence being 
applied to "interpret" or **recogni2e" specific information 
embedded within the document The structural linkages are 
usually defined as accepted information storage or inter- 
change formats and vary in complexity from simple linear 
data strings in which numbers arc stored in a one-dimen- 
sional forward-reading sequence, through formats such as 
rich text format (RTF) or symbolic link format (SYLK) files 
representing data and particular visual attributes for display- 
ing or representing that data on a page, to page description 
languages such as Postscript in which an electronic image is 
reduced to and expressed exclusively as mathematical for- 
mulas, callable subroutines, or vectors representing indi- 
vidual components fix»m which the electronic image can be 
interpreted and reconstituted for display or representation. 

Two distinct standards have developed forjudging the 
threshold for integrity and reqiusite precision in the trans- 
mission, storage, and retrieval of electronic content versus 
electronic images. 

The integrity of electronic content is based on a quanti- 
tative threshold, and complete or error-free precisioii is 
firequentiy presimied. Lack of quantitative precision may 
have varying degrees of impact on the qualitative integrity 
of electronic content. For example, a single discrete or 
one-bit error in an RTF file may have only a minor effect on 
the textual representation of a character, and therefore a 
negligible effect on the qualitative integrity of that file when 
displayed or represented as a page; the same numerical error 
in a SYLK file may disrupt a localized portion of the matrix 
such as a row or colunm in a spreadsheet, but leave a large 
portion of the spreadsheet intact and imaffected from a 
qualitative standpoint; a one-bit error in a program file could 
be fatal to the program's operation and therefore completely 
destroy all qualitative integrity. Consequently, preventing 
even one-bit errors in the transmission, storage, and retrieval 
of any electronic content document is assumed to be mission 
critical because there is an expectation of absolute quanti- 
tative precision (and therefore complete qualitative integ- 
rity) associated with electroiuc content 
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Conversely, relatively significant quantitative "errors" 
may be introduced into electronic images and yet be 
accepted, because they leave the content and form of the 
electronic image efiFectively intact without approaching the 
threshold established for qualitative integrity in the clec- 5 
tronic image domain. Again, it should be noted that thresh- 
olds for qualitative integrity in the electronic image domain 
are currently set at artificially low levels because they are 
being established or judged as a function of the human visual 
comprehension of presentations (raster or LCD displays) 
and representations (paper or film printouts) of the corre- 
sponding image. 

As discussed below, the conventional wisdom in interac- 
tive document processing is to capture an image from a 
source document, and to keep that electronic image exclu- 
sively in the digital domain once it has been digitized. It is 
commonly believed that maintaining the digital namre of the 
electronic image preserves the accuracy and integrity of its 
informational content. In fact, this is a mistaken assumption. 

First, it must be remembered that any digitization process 
relies upon an initial transition from the analog to digital 
domains, and there is an equal probability of introducing 
errors in informational content at thai step as there would be 
for any other subsequent analog-to-digital conversion. That 
is, once an image is digitized it is relatively simple to verily ^ 
that the "data" content does not subsequently change, how- 
ever there is no assurance that tiie origina] "data" accurately 
reflects the true informational content of the source docu- 
ment. Added to this is the problem that in order to obtain 
reasonable transmission and storage times for most docu- 
ment processing, images are routinely captured and pro- 
cessed at bit-4epths much less than 8-bits grayscale, and 
often as monochromatic images of very low resolution. 
Thus, a great deal of informational content is being inten- 
tionally discarded for no reason other than to facilitate 
subsequentiy digital processing techniques. The quantity of 
"errors" that might be introduced by repeated analog-to- 
digital and digital-to-analog conversion of an electronic 
image still remain many orders of magnitude below the 
quantity of "errors" that are interjected into images by ^ 
current document processing technologies. 

Second, if the capture device defines the rate limiting 
operation in the signal conversion process, the technology 
disclosed herein provides informational integrity equivalent 
to exclusively digital processes. If the storage device defines 45 
the rate limiting operation, and the caplure/storage/retrieval/ 
transmission pathway involves four conversions between 
analog and digital signals, the system has only increased the 
probability of losing a "pixel" by three times when com- 
pared to an exclusively digital process. Since the informa- 50 
tional content is several orders of magnitude higher than a 
conventionally processed digital image, a three times 
increase in the probability of an en*or in one "pixel" is 
insignificant. It should also be remembered that this discus- 
sion focusses on the complete loss of a pixel, whereas in an 55 
analog frame of reference an error might actually amount 
only to shifting a pixel to a slightly higher or lower gray 
level. Isolated errors of this type would be completely 
imperceptible when imbedded in an image comprising more 
than a million adjacent pixels, with each pixel having two 60 
hundred or more possible discrete grayscale values. The 
actual efl'ect of an error of tiiis type is further reduced when 
one considers the potential for applying oversampling and 
signal averaging techniques for each analog-to-digital con- 
version. 55 

The application of hybrid processing may thus be equated 
with a tradeoff between the sanctity of bit- value integrity for 



the sake of vastly increased processing speeds. However, the 
loss of bit-value integrity is relatively meaningless when 
applying this technology, because the techniques for main- 
taining reasonably high precision in analog-to-digital con- 
versions reduces the frequency at which eixors may occur, 
and any error becomes statistically insignificant if one 
considers the many-order increase in the magnitude of 
informational content when using greater bit-depths that 
effectively dilute any error. 

The use of 8-bit grayscale is preferable for most current 
interactive document processing operations since it provides 
a displayable image having far superior informational con- 
tent than conventional techniques for storing tangible source 
documents (which often treat the original source document 
as line art and rely on conversion to monochromatic or 
one-bit levels which discards the majority of the document's 
actual informational content.) It has been shovra that an 8-bit 
grayscale image is acmally more readable than a lower level 
or monochromatic image corresponding to the same docu- 
ment, due primarily to the wealth of informational content 
situated in the zone situated above the full content of a lower 
level image and below the portion of the 8-bit grayscale 
image that is beyond the limits of visual perception. As such, 
there is a large portion of a true 8-bit grayscale image that 
cannot be visually perceived without the aid of enabling 
equipment, and which would be discarded in normal view- 
ing. The ability to display an electronic image in 8-bit 
grayscale therefore also permits a wide range of user- 
definable adjustments in tiie contrast levels and grayscale 
filters to be applied to the viewed image that facilitate 
interactive discrimination or enhancement of the informa- 
tional content in the image for certain applications (such as 
diagnostic review, of X-rays, MRI scans, or other medical 
images) without affecting or altering the source document or 
its stored image. 

In addition, the 8-bit grayscale electronic image will 
contain background details such as security paper patterns, 
watermarks, illegible color inks, maricings that are faded or 
too faint to be perceived by viewing die original source 
document, as well as creases, smudges, stains, and other 
unique identifying details that assure far greater certainty 
when verifying the integrity and authenticity of tiie elec- 
tronic image, far exceeding the currentiy accepted standards 
for duplicates of financial, business, and legsJ records. Tlie 
use of 8-bit grayscale also permits the capture of electronic 
images from damaged or aged tangible source documents 
such as burnt papers or faded microfiche that could not be 
reproduced by other means, and which may be accom- 
plished as if the original source documents were xmdam- 
aged. 

Because electronic images currentiy exist as bitmaps or 
digital arrays in semiconductor memory, they are transmit- 
ted, stored, and retrieved in digital form and these operations 
do not themselves introduce quantitative errors in the con- 
tent of the electronic images. The most significant "errors" 
or losses in content generally result from intentional trans- 
formations occurring during processing steps such as digi- 
tizing a tangible document into an electronic image, or 
formatting an electronic image for mtegration into a pro- 
gram or as part of a file storage protocol. 

It must be remembered that the acceptable threshold for 
qualitative integrity of electronic images as used in reference 
to Uie technology disclosed herein is several orders of 
magnitude greater than can be perceived by human vision 
when viewing a tangible document, and is at or near the 
limits of what can be practicably achieved by conventional 
representation devices such as laser printers or film record- 
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ers. Consequently, while the technology disclosed herein tangible and electronic content domains, however virtually 
will normally be implemented in embodiments which pro- all technology now in use relies on some intermediate 
vide less than the absolute or complete quantitative precision transition through the electronic image domain, 
associated with conventional digital modalities for purely xhis terminology can be somewhat misleading. For 
practical reasons, the qualitative integrity of documents 5 example, a raster is conventionally defined in electronics as 
created in or converted to the electronic image domain and a uniform rectangular pattern of scanning lines having an 
subsequently transmitted, stored, and retrieved by the dis- aspect ratio determined by horizontal and vertical synchro- 
closed modality will exceed that currently accepted for nization and timing (or blanking) pulses that is produced on 
document processing in the electronic image domain and yet a cathode-ray tube (CRT) in the absence of a modulated 
permit substantial decreases in the lime required to complete lO signal. In image processing, however, a raster usually means 
those processes. the display of the digital array associated with an electronic 
The methods disclosed herein may be applied equally to image on a raster device such as a monitor, which could as 
documents in either the electronic image or electronic con- easily be displayed or projected directly as a bitmap using an 
tent domains. However, the use of this modality for pro- LCD or LED device. Furthermore, the scanning lines in a 
ccssing electronic content will be only be adopted if a raster have no relation to the process of scanning a tangible 
sufficient threshold level of quantitative integrity (as defined document in most devices that are called scanners, which 
by the particular application involved) can be consistently conventionally incorporate line- or area-array CCD technol- 
maintained or verified. The complexity or expense associ- ogy. 

ated with assuring this requisite level of quantitative integ- it may be readily appreciated that an electronic image 
rity for electronic content may be commercially prohibitive 20 exig^g as a digital array in memory and does not require 

given the adequacy of conventional systems now used being displayed as a raster or bitmap, but such a presentation 

extensively for digital transmission, storage, and retrieval of is merely an aid for human visualization and comprehension 

electronic content, despite the significant differential in of the electronic image as it resides in memory. The display 

speeds at which the technologies would operate. is therefore a ''virtual" document and the bitmap or digital 

Furthermore, at present the vast majority of penonal and array is the tme or "original" document. The process of 

business communication is conducted using documents that transposing a document from the electronic content domain 

remain almost exclusively in the tangible or electronic to the electronic image domain really constitutes mapping 

image domains, and tiie immediate need for applying this the image into a digital array in memory, thus the term 

modality to electronic images far supersedes the compara- **bitmapping" has been added to the conventional nomen- 

tively insignificant demand for accelerating the processing clature for this transition. 

of documents in the electronic content domain. Application Similarly, transposing a document from the tangible 

of this modality to documents created in or converted to the domain to the electronic image domain requires the same 

electronic image domain provides an effective precision that niapping of an image into a digital array in active memory, 

is functionally indistinguishable from complete qualitative and could just as well be termed "bitmapping." In the field 

integrity, and therefore substantially greater than basic levels of document processing, where an operator works at a 

of qualitative integrity now utilized for tangible and elec- computer or workstation, the term "scarmer'* has tradition- 

tronic image documents. In addition, the complexity and ally been applied to a peripheral capture device which 

expense associated with this modality are no more than for creates a digital army or bitmap of a tangible document, and 

existing technologies. that digital array is simply "dumped" or swapped into a 

For tiiose reasons, the remainder of this discussion will segment of active memory within the computer. Conversely, 

focus on documents in the tangible and electronic image if die peripheral capture device produces an analog output of 

domains, however it is understood that the modality may be sequential frames, the transition to a digital array or bitmap 

readily applied to the electronic content domain if adapted or may be performed by the process cominonly called "frame 

augmented to provide suitable assurances that acceptable grabbing** either by the peripheral device or on board the 

quantitative precision can be consistently maintained or computer. In this case, the term "capture" is utilized to 

verified throughout the transmission, storage, and retrieval include botii the processes of scanning and frame grabbirig 

processes. where a digital array or bitmap of an electronic image is 

Since the focus of this discussion is on particular embodi- produced and resides in active memory, 
ments designed for the tangible and electronic image 50 As previously noted, the transition between any two 

domains, it is presumed that any "data" embodied witiiin a domains almost always results in some transformation of the 

tangible 'or electronic image exists as a function of the original document to a new or derivative document, whether 

document' s content and form. Therefore, any data contained or not that transformation is visually perceptible. Similarly, 

in a document is subject to comprehension and recognition any presentation or representation of an electronic image 
by die visual inspection of presentations or representations 55 either produces a virtual image or creates a new tangible 

of the image by humans, who may then manually transcribe document. Theoretically, the virtual image and the new or 

or encode diat data for use as electronic content, or by die derivative document should be identified and treated as new 

application of artificial intelligence to recognize, interpret. documents having different informational content and form 

and encode tiiat data from within die image. than the original document residing in memory as an elec- 

Ttiere have traditionally been ifour additional operations 60 Tronic image, 

associated with transposing or bridging a document from Many factors affect the degree to which the informational 

one domain to another: scanning or capture (tangible to content in a presentation, representation, or transformed 

electronic image); recognition (electronic image to elec- image diverges from that of the original docuinent. Because 

tronic content); rasterization or bitmapping (electronic con- informational content is judged as a function of visual 
tent to electronic image); and output or marking (electronic 65 recogrution, three factors have beconie basic to measuring 

image to tangible.) These operations do not encompass two the informational content of an image: kind, depth, and 

possible transitions which could occur direaly between the density. 
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Kind designates the classification of the image, and for 
purposes of this discussion may be monochromatic, gray- 
scale, or color. These three kinds of images encompass most 
or all of the visually perceptible documents. At the same 
time, it should be remembered that there are other types of 5 
documents (and certain information within otherwise visible 
documents) that may only be perceived with some type of 
enabling technology. Inftared and ultraviolet represent two 
familiar examples where enabling technology produces 
images containing informational content that is not other- 
wise visually perceptible, but it should be remembered that 
true grayscale also contains large quantities of informational 
content that may be otherwise discarded in human visual- 
ization. 

Depth is a digital measure of the quantity or '*bits" of 
information associated with each informational bundle or 
picture element ("pixel.") The most frequently used depths 
in interactive doctmient processing are one-bit (effectively 
monochrome), 4- and 8-bit for grayscale, or 8-, 24-, and 
32-bit for color. 20 

Density is the physical spacing of informational bundles 
in pixels per unit measurement Density is irrelevant to an 
electronic image, and only becomes a factor to consider 
when presenting or representing an image. Density is often 
interchanged with resolution, and different standards and 25 
references have developed for display resolution and print- 
ing resolution. However, resolution is really a function of 
image comprehension as determined in the visual frame of 
reference. When the term resolution is used in interactive 
document processing, it is being used as shorthand for the 30 
"absolute resolution" which is the minimum separation 
between pixels or informational bundles that may be distin- 
guished or resolved. 

Density and resolution are important terms in document 
processing because they permit operators to specify an 35 
acceptable level for representing or presenting a document 
They are also easily confused when comparing representa- 
tion resolution with presentation resolution. For example, 
referring to FIG. 8, one might specify outputting an elec- 
tronic image such as a continuous black-to-white 8-bit 40 
grayscale gradient to tangible form on a 300 dots-per-inch 
(dpi) laser printer for one use, but a much higher resolution 
for another. These specifications are further complicated by 
the fact that most tangible output devices are monochro- 
matic, and grayscales are simulated or approximated by 45 
applying a selected halftone sca-een to the image, or printing 
the image as a dither pattern to approximate levels of 
grayscale. The halftone screens are usually denoted by the 
frequency (number of lines per inch) and the angle of 
orientation. In any case, the true resolution of the output 50 
device remains constant while the effective density of the 
image changes, and the actual informational content of the 
outputted docimient decreases compared to the original 
electronic image. A four bit grayscale gradient would there- 
fore theoretically contain 16 shades of gray including black 55 
and white, but if printed at 300 dpi would show only about 
12 gray levels due to the type of dithering pattern used and 
the processor's calculation of the optimal number of steps to 
create a smooth blend or transition between levels. As such, 
in the example recited above, the 8-bit grayscale gradient 60 
printed at 300 dpi resolution may result in approximately 58 
visibly discemable gray levels or less (FIG. SB) with the 
dots of the dithering pattern being very apparent, whereas 
the same gradient printed at a 3360 dpi with a 150 line 
horizontal screen will produce the same number of gray 65 
levels (FIG. 8C) but at a much higher resolution. As such, 
specifying a higher resolution may achieve a great increase 
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in informational content without increasing the grayscale 
depth, and increasing the grayscale depth may also greatly 
increase the informational content without requiting higher 
resolution. This may be compared with a true 8-bit image 
viewed on a monitor in which each "pixel" is displayed at 
one of 256 discrete gray levels, and which has a resolution 
on the order of 70-80 pixels per inch. It may be appreciated 
that increasing both resolution and grayscale depth will have 
a corresponding impact on informational content. 

For purposes of creation and transformation in the elec- 
tronic image donoain, an image is usually treated as being 
composed of bitmaps, objects, paths, models, or renderings. 
Bitmaps arc created and transformed by altering the char- 
acteristic value assigned to individual pixels within the 
bitmap. Objects and paths are transformed by altering either 
the fundamental definition of the object or path, or a 
characteristic attribute associated with that object or path. 
Attributes may be very simple or extremely complex, and 
attributes of paths may depend upon linkages and relation- 
ships to other paths and their attributes. Objects are gener- 
ally self-contained. Models are the three dimensional 
equivalent of objects, but are composed of one or more 
assembled structural blocks. A rendering is essentially a 
complex bitmap created by applying attributes to a model, 
but which cannot be interactively transformed by altering 
the characteristic value of separate pixels. 

Objects, paths, models, and many renderings are usually 
"device density" or "output resolution" dependent, meaning 
that they are treated within an interactive program as for- 
mulaic expressions each having a set depth but variable 
density, and when presented or represented they will adopt 
the highest density afforded by the capabilities of the pre- 
sentation or representation device. For example, a simple 
arcuate path expressed in Postscript language will have a 
shape, size, and a specified value within the range dictated 
by the image's grayscale depth. When output on a 300 dpi 
printer, the image will have the same basic shape and size as 
the electronic image, but the printer will utilize its 300 dpi 
density to provide the best approximation or effective reso- 
lution of the grayscale value and path contours defined by 
the formulaic expression as possible. Output on a 1250 dpi 
printer will again have the same shape and size but higher 
density, meaning that the effective resolution of the gray- 
scale level and path contours will more closely match the 
formulaic expression. Effective resolutions may be so low 
that losses in informational content are clearly perceptible, 
or so high that they exceed visible comprehension. The 
nature and use of the tangible document being represented or 
the virtual image being presented will dictate the preferred 
or acceptable effective resolution. 

The three measures of the informational content in an 
electronic image are completely independent of the image's 
physical size. As a practical matter, limitations imposed by 
the processor speed, available memory, and storage medium 
in a document processing system will sometimes require 
reducing an electronic image's depth or density as its 
physical size increases. Kind, depth, and density may also be 
selectively manipulated to achieve a particular visible or 
perceptible result when an electronic image is transformed, 
represented, or presented. 

Advances in both interactive and non-interactive docu- 
ment processing are evaluated according to five criteria: 
compatibility, transparency, decentralization, modularity, 
and operational capacity. 

Compatibility refers to the capability of different tech- 
nologies to utilize the informational content of a document 
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Compatibility is now limited to storing a document in one or 
more predefined formats, with interactive document pro- 
cessing programs having the ability to access information 
only from specific formats. Usually, the higher the level of 
a program the more formats in which it will store and 
retrieve a document. If a format or conversion is unavail- 
able, all or a portion of the informational content of the 
document will be inaccessible. For the most part, compat- 
ibility in transmission is limited to modem and network 
protocols for electronic content and facsimile and network 
protocols for electronic images. 

Transparency has two definitions. At one level, transpar- 
ency is the movement of documents between domains 
without loss of informational content. Operational transpar- 
ency is the ability of an user to employ a technology without 
conscious consideration of the inherent transformations pro- 
duced by that technology. In interactive document process- 
ing, effective transparency can be defined as the transition 
from the tangible to electronic image domain without a 
visually perceptible loss of qualitative integrity, and as 
equivalent access to electronic image documents through 
transmission or retrieval processes independent of the loca- 
tion of the original document and without regard to inter- 
mediate transformations. 

Decentrahzation refers to the ability of users of conraiu- 
nicating systems to perform the same document processing 
operations on the same documents, and to have the same 
capabilities that are available to operators at a central 
document processing or coordination facility. 

Modularity refers to the linking of single- or multi- 
function devices for document processing. Modularity 
increases the functions performed by devices, or increases 
the available linkages between devices, to optimize paths 
through which documents are processed. For example, 
merging a scanner with a printer to accomplish the functions 
of scanning, photocopying, facsimile reception, and printing 
is a higher order of modularity than having four separate 
devices performing the same four functions. Another facet of 
modularity is scalability, which permits the addition (or 
subtraction) of a redundant peripheral device to a system to 
increase (or decrease) a particular operational capacity of the 
system without requiting replacement of the complete sys- 
tem. For example, the addition of a SCSI hard drive to a 
personal computer pennits the system to be scaled upwardly 
to increase its information storage capacity. 

Operational capacity for most interactive document pro- 
cessing is determined by measured capabilities or bench- 
marks such as processor dock speeds (in megahertz), mil- 
lions of instructions performed per second (MIPS), bit depth 50 
of semiconductor memory, access times for semiconductor 
memory (in nanoseconds), transmission rates (in baud), 
storage capacity (in megabytes), storage density, and seek 
and read/write speeds (in milliseconds to microseconds). 



that have been unacceptable for interactive document pro- 
cessing applications (particulariy in transmission or storage 
and retrieval operations), and selective storage and retrieval 
capabilities have not been developed in the analog fonm. 
5 In discussing the speed of transmission, storage, and 
retrieval of documents created in or converted to the elec- 
tronic image domain, it is important to bear in mind that this 
modality is concerned only with electronic images that are 
expressed as a bitmap or digital array in semiconductor (or 
10 other) memory that will be utilized for the selective trans- 
mission, storage, and retrieval by an operator. 

The constraints in the speed at which electronic image 
documents may be transmitted, stored, and retrieved result 
from the application of technologies designed primarily for 
electronic content to all electronic domain documents, and 
the inherent nature of the digital technologies used to 
process electronic content. For example, conventional com- 
pact discs or magnetic hard disk drives will have seek times 
on the order of milliseconds, and read/write times on the 
order of microseconds per digital word. This is approxi- 
mately two to three orders of magnitude slower than the 
routine transmission, storage, and retrieval rates for the 
technology disclosed herein, which operates corresponding 
to the nanosecond time range that conventional static ran- 
dom access memory (SRAM) is capable of exchanging bit 
groups. 

The routine transmission, storage, and retrieval rates of 
the technology disclosed herein wiQ be positively affected 
by the utilization of memory having faster access times, as 
well as the capability to process larger digital words. The 
limiting factors become the speed at which sequential access 
semiconductor memories (such as CCDs) can be operated as 
the size of the digital arrays increase, and the signal 
exchange rates at which optic^ and magneto-optical storage 
media can perform. 
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A dilemma has existed since the advent of digital pro- 
cessing for electronic images. 

First, digital signal processing is inherentiy slow, and 60 
designers have been unable to obtain high resolution at fast 
speeds using digital technology. Second, analog signal pro- 
cessing is fast, but designers have been unable to produce 
and process high resolution images or provide the ability to 
interactively interface with electronic images in analog 65 
form. Furthermore, analog signal processing presents prob- 
lems such as signal loss, degradation, noise, and distortion 



Description of the Prior Art 

The modality disclosed herein may be applied in either 
one of two forms: (1) the conversion of tangible documents 
to electronic images and the transmission, storage, and 
retrieval tiiereof; or (2) the creation or transformation of 
documents in the electronic image domain and the trans- 
mission, storage, and retrieval thereof. In the preferred 
embodiments for document processing in business environ- 
ments, an integrated system having both capabilities oper- 
ating in tandem is disclosed. There are two equally satis- 
factory views of this integrated system. If considering the 
first form, one could view the system as a four step process 
comprising the steps of source capture, image digitization 
and handling, image storage, and image retrieval. Con- 
versely, in considering the second form one could view the 
system as comprising a core process of image handling 
(holding and manipulating a bitmap in active memory), 
image storage or transmission, and image retrieval, with the 
steps of source c^ture and digitization being ancillary or 
optional processes utilized only when converting tangible 
documents to the electronic image domain. 

The majority of interactive document processing occurs in 
a user- or operator-interactive environment which may be 
referred to generally as interactive document processing 
(IDP). Almost all interactive document processing is per- 
formed using a computer as the interactive processor. The 
computer may be a dedicated device such as a word pro- 
cessor, a personal computer, workstation, or a computer or 
terminal linked to a file server or mainframe by a network. 
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In these systems, once an image is created in or converted 
to the electronic image domain as a digital array or bitmap, 
it remains in digital form throughout all subsequent opera- 
tions even when the digital airay is transmitted, stored, or 
retrieved. 

Preservation of the digital format is maintained for two 
reasons. First, the electronic image is being handled as 
electronic content for purposes of storage and retrieval 
because current storage methods do not differentiate 
between electronic content and electronic images, and abso- 
lute quantitative precision is dictated for the electronic 
content including programs and data files. Electronic content 
is not separated from electronic images, and one device is 
utilized simultaneously for the storage of both electronic 
content and electronic images. The value of the depth for 
each pixel of an electronic image is therefore also stored on 
the available medium in digital form as though it were 
electronic content. Hiis preservation of the digital format for 
transmission, storage, and retrieval of electronic images has 
remained almost exclusive and inviolate since the introduc- 
tion of semiconductor memory. Only recently have other 
means of recording and replaying electronic images been 
introduced for use in multimedia processing, however these 
means have no applicability to the transmission, storage, and 
retrieval of electronic image documents. 

Second, digital transmission systems provide better signal 
to noise ratios resulting in higher efficiency for a set preci- 
sion. (Purely electronic systems are conventionally not 
measured according to precision, but rather by the efficiency 
achieved by in a system given a predetermined precision 
threshold and a set transmission configuration.) As such, 
digital transmission provides very high efficiency when the 
required precision is nearly absolute, but is extremely inef- 
ficient when compared to analog transmission for precision 
levels only slightly below absolute. Since absolute precision 
is presumed for transmission, storage, and retrieval of elec- 
tronic content, digital transmission is used for both elec- 
tronic content and electronic images. 

The result has been to apply digital tedmology wherever 
possible to produce inter-device compatibility (for example, 
between telecommunications systems, facsimile transmis- 
sion devices, and personal computers), and the demand for 
digital components used to transmit, store, and retrieve 
information has resulted in commercial devices which are 
readily available and inexpensive. In fact, the prevailing 
trend has been to adopt digital systems for many other 
information processing and recording systems such as 
audible telephone communication, compact disc recording, 
and digital audio tape (DAT). These digital technologies are 
not without many real merits and advantages that may 
readily be appreciated by both the technology designers and 
users of the particular consumer devices, and there has been 
a corresponding shift among product designers and consum- 
ers towards prefeiring digital technology for purely aesthetic 
reasons. 

Examples of the proliferation of digital technologies for 
storing and retrieving electronic images include many of the 
CD-ROM drives and applications that are commercially 
available for personal computers, the development of the go 
Kodak Photo-CD standard currendy being implemented for 
transferring 35 mm photographs directly to compact disc, 
and the use of rewriteable optical and magneto-optical disc 
recorders for file storage. 

Even though it is known that the qualitative integrity of an 65 
electronic image may not be perceptibly diminished by the 
utilization of an analog format, no technology has yet been 
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developed for the transmission, stomge, and retrieval of 
electronic images in the field of interactive document pro- 
cessing. 

Outside the field of interactive document processing, 
there are some representative examples of new develop- 
ments concerning the application of analog technology to 
information processing. 

Efforts to produce major developments in analog tech- 
nologies for television and broadcasting have been focussed 
on the development of high-definition television systems, 
the production of larger array CCD cameras, the utilization 
of satellite transmission, and the use of optical laser discs for 
recording. 

In the field of radar scan conversion, various processes 
have been applied to increase the rate at which polar 
coordinate maps can be processed and converted to an X-Y 
memory map and displayed using raster devices. U.S. Pat. 
No. 4,754,279 to Cribbs discloses a scan converter which 
accomplishes the conversion of an analog radar signal to a 
digital array through two high speed memory buffers and an 
intervening segmentation of the electronic image in active 
memory, and the subsequent assembling and display of that 
army as a raster image corresponding to the bitmapped 
electronic image. Similar polar to X-Y scan converters are 
also fotmd in ultrasonic and other medical imaging devices, 
with a representative example being U.S. Pat No. 4,275,415 
to Engle. 

Scan converters related to those disclosed in the Cribbs 
*279 and Engle '415 patMits have also been utilized with 
certain modifications in systems to scan tangible documents 
and store the corresponding electronic images as firames in 
a conventional video tape format. These systems may be 
interfaced with a conventionai personal computer, and rep- 
resentative examples of such systems include those currently 
marketed by Folsom Research of Folsom, California and 
RGB/Spectrum of Alameda, Calif. 

Although the technology disclosed in the Cribbs *279 and 
Engle '415 patents and die Folsom or Spectrum devices 
permit operators to record sequences of electronic images at 
rates exceeding those for current digital components, they 
do not provide the required capability for transmitting, 
storing, or retrieving electronic images. 

In particular, the technology related to radar scan con- 
verters is directed to monochromatic images, and involves 
certain operations for correlating discrete segments of 
adjoining raster lines to the corresponding pixels in a digital 
bitmap. The current technology also relies on using a 
nibble-mode for reading and writing information from high 
speed buffers to update only the pixels in the digital array 
corresponding to radar pixels that are activated or faded as . 
a target moves, and this technology does not translate to 
grayscale or color images having far greater depth, nor to the 
transmission, storage, or retrieval of complete electronic 
images. 

The commercial devices that may be interfaced with 
document processing systems further do not permit indexing 
and acquisition of selected images at workable rates for 
interactive document processing operations, but rather rely 
on replaying instead of retrieving documents. These devices 
utilize the broad bandwidth obtainable with magnetic tape 
mediums for recording electronic images having greater 
than one-bit depth in a format that corresponds to a video 
"frame," In comparison, the storage and retrieval of elec- 
tronic images at Mgh rates (in tens of images per second) and 
large volumes (tens of thousands of images per unit of 
medium) requires indexing and selective acquisition of 
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Stored images on a medium such as an laser or optical disc, 
which has fixed bandwidth and limited frame composition. 

Retrieval and display of stored electronic images at 
roughly comparable rates (one or more images per second) 
and volumes (fifty thousand images per unit of medium) are 
currently accomplished on rewriteable optical disc recorders 
such as the Panasonic LQ-4000 Erasable Videodisc 
Recorder, however this retums the storage and retrieval 
operation to the digital format, and current systems only 
permit limited operation for capture and storage before 
encountering downtime for processing. 

The application of hybrid signal processing to accomplish 
signal compression for serial recording is also known to the 
art. 

U.S. Pat. No. 5,006,936 to Hooks discloses a method of 
compacting a analog signal corresponding to unframed 
continuous-signal data output wherein an instrument's ana- 
log output signal is converted to a digital signal, and portions 
of the digital signal are alternately written to two memory 
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U.S. Pat. No. 4,378,571 to Handy discloses a technique 
for increasing the field of view of a flatbed seamier while 
maintaining the device's ori^nal resolution by using two 
CCD chips fixed in an array having a specified displacement 
relative to one another that corresponds to one half of the 
total field of view. The analog output signals of each CCD 
are sampled and averaged, and the gross grayscale levels for 
the two halves are equalized with one another by amplifying 
the DC gain level for one or both of the signals toward a 
common level. The signals are then processed by an analog- 
to-digital converter to digitize the image. 

This technique is essentially a sunplified version of 
"ganging'* smaller CCD chips to produce a larger X-Y pixel 
matrix, wherein a first row of pixels are read sequentially 
from one CCD chip and the aligned first row is then read 
from the second CCD chip, followed by the second pair of 
rows from each CCD chip, and so forth to produce a 
"horizontal'* bitmap. It is also possible to simply read all the 
rows of one chip followed by all the rows of the second, to 



buffen. As the (Sgjtal signal is being written to one memory 20 produce -a "vertical" bitmap. Ganging more than two CCD 



buffer, the other memory buffer is simultaneously read at a 
faster rate than it was written. The compacted digital signal 
is then converted to an analog signal having a greater 
bandwidth than the original data output signal, and the 
appropriate synchronization signals are applied correspond- 
ing to a conventional NTSC video format The compressed 
and NTSC-synced signal is then stored using conventional 
video equipment such as a magnetic tape or optical laser disc 
recorder, A decoder circuit is utilized to strip the synchro- 
nization signals and decompress the signal back to the 
original unfiled continuous-signal data output for serial 
replay at the same rate as the original data was produced. 
The Hooks *936 device therefore compresses a one-dimen- 
sional time-variable signal by increasing its bandwidth suf- 
ficiently that synchronization pulses may be added to fill the 
gap between the compressed signal and a standard NTSC 
video format, and then utilizing that NTSC equipment for 
recording and playback. 
Methods for compressing analog signals using hybrid 
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chips in each direction results in a combination of both 
horizontal and vertical bitaiapping between different CCD 
chips. This technique was used in video cameras when CCD 
chips having only a 512x512 X-Y pixel matrix were avail- 
able, however cameras using CCD chip matrices of 1024x 
1024 and 2048x2048 pixels are now readily available (and 
could similarly be ganged to produce even larger matrices.) 
The principle of the Handy *571 patent has appaiendy also 
been applied to hand-held scanners which are used to 
sequentially scan parallel strips of a source document, 
wherein the speed at which the timing wheel within the 
scanner is rolled across the document provides a crude 
physical control or reference for the number of pixel rows- 
per-inch that are scanned (or conversely, the ratio by which 
one strip is scaled along its length to match the pixel 
rows-per-inch of another strip.) 

As such, the Handy '571 patent does not teach an actual 
"seaming" process for reconstituting divided images, and it 
also points out the inherent enors in the informational 



signal processing similar to that disclosed in Hooks * 93 6 40 content of an image that may be surreptitiously introduced 



have been utilized for many years. While the Hooks '936 
device provides an effective system for the recording and 
playback of large spans of unframed continuous-signal 
transmissions such as flight recorder data using lower cost 
NTSC video equipment, compaction methods such as dis- 45 
closed in Hooks '936 are not applicable to the selective and 
non-sequential transmission or storage and retrieval of two- 
dimensional electronic images. First, the compression 
method described in Hooks '936 is most suitable for mag- 
netic tape recording which may accommodate and utilize the 50 
comparatively unlimited bandwidth of video tape, whereas 
the bandwidths associated with the information^ content of 
two-dimensional electronic images (particularly those hav- 
ing conventional grayscale or color depths used in interac- 
tive document processing) are akeady greater than NTSC 55 
equipment would permit, and signal compression would 
only increase the bandwidth. Second, compaction methods 
such as disclosed in Hooks '936 permit only serial playback 
of continuous-signal data output Third, continuously alter- 
nating the flow of signal to or firom two separate memory 60 
buffers prevents utilizing those memory buffers at the junc- 
tion with an interactive document processing interface. 

Various references in the prior art have discussed "stitch- 
ing" techniques used to combine the outputs from two CCD 
arrays for use in a flatbed scanner, and for permitting the use 6S 
of NTSC display devices for viewing high definition tele- 
vision (HDTV) broadcasts. 



on a very large scale when using conventional digital 
techniques for document processing operations. The. Handy 
'571 patent further does not teach any of the time-based 
correction and masking principles necessary to make the 
actual seaming of a divided image possible, nor to overcome 
time-based errors introduced by optical laser disc recorders 
and other devices. The Handy '571 device also places the 
analog multiplier used for gain correction upsteam of the 
analog-to-digital converter used for digitization, and there- 
fore reinforces the existing assumption that conversion to 
the digital domain is a priori necessary for image processing. 

U.S. Pat. No. 4,899,220 to Basile discusses a technique 
for permitting the center portion of a wide aspect ratio high 
defiiiition television (HDTV) broadcast to be viewed on a 
conventional NTSC display, and alternately a display appa- 
ratus used in combination with a conventional NTSC display 
for viewing the entire wide aspect ratio image. The Basile 
'220 patent teaches dividing lie scan lines into segments 
corresponding to a center portion of a wide aspect display 
and two side panels, converting to the NTSC format for 
broadcasting, and reassembling those segments after broad- 
casting. However, this procedure produces visible lines 
vertically , bisecting the display due to dot crawl. Conse- 
quentiy, Basile '220 also teaches randomly varying the 
location of the stitch points between the line segments of 
adjacent lines or between fields, thus scattering the stitch 
points throughout the image and varying their location 
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between fields to a degree where the stitch points will not 
produce a visible pattern interrapting the image when dis- . 
played at 30 frames per second. 

It may be readily appreciated that Basile '220 does not 
leach eliminating the visible stitch points, but merely relo- 5 
eating them so that a viewer will not perceive them when 
viewing a many changing images displayed each second. As 
such, the technique of Basile ^220 cannot be applied to 
document processing operations where the introduction of 
stitch points would still degrade the informational content 
and undermine the integrity of a stored image. Even though 
the degradation might not be visible when displaying many 
changing images, some degradation would be visible when 
viewing a static display of the same image, which would 
render the system unsuitable for many operations such as 
medical imaging. Furthermore, since tbeBasile *220 process 
relies on random distribution of stitch points among unre- 
lated images, the effects of these degradations would accu- 
mulate rapidly when repeatedly storing and retrieving the 
same image, to the point where the image could be effec- 
tively destroyed over time. While the technology of Basile 
*220 is suitable for raster display purposes where the camera 
or broadcast generator has a very stable output, analog 
storage devices such as laser disc recorders or players have 
significant time-based errors in their output signals which 
cannot be corrected for using the technology disclosed in 
Basile '220, just as it does not provide any time-based 
correction to prevent the visible stitch points. The Basile 
'220 and Handy '397 patents are therefore similar in the fact 
that they do not give any consideration to absolute timing 
relative to an external reference. 

Moreover, while the masking technique accomplished by 
varying the time at which line segments are switched in 
controlled amounts is a strong feature when considering a 
displayed broadcast image, the method disclosed in the 
Basile '220 patent apparently only permits manual control 
over image content without providing any grayscale cali- 
bration, DC matching, gain adjustment, or color control 
between adjacent line segments or on a line-byline basis. In 
addition, the Basile '220 device (with or without the ability 
to display the side panels) is still limited to the conventional 
resolution of the display, which in this case is the normal 
NTSC resolution. 

As such, technologies of the type disclosed in Basile '220, 
Handy '571, Hooks '936, and many other references in these 45 
distinct fields of art provide very good solutions to particular 
problems associated with converting between display or 
recording formats, ganging capture devices to increase the 
effective field of view, and the like. 

However, it is believed that the prior art does not teach, so 
suggest, or even address the much broader concept of 
interactive document processing in an hybrid (analog/digi- 
tal) domain utilizing techniques such as multiplying the 
effective resolution of a capture device, segmenting the 
existing bitmap of an electronic image to optimize its 55 
informational content versus its transmission, storage, or 
retrieval times for a given operation or existing apparatus, 
storing an electronic image having a predetermined infor- 
mational content on randomly accessible storage mediums 
having fixed bandwidths which are smaller than those asso- 60 
dated with the original image, transmitting an electronic 
image via a transmission pathway similarly having a limited 
bandwidth smaller than that associated with the original 
image, integrating these processes in an interface which 
permits interactive utilization of an electronic image and 65 
transparently passing the image back-and-foith between 
conventional low speed digital processing pathways and 
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high speed analog processing pathways, nor the many inde- 
pendent advantages and synergistic effects achieved by 
applying these concepts as part of a complete document 
processing environment, network structure, or communica- 
tions system. 

BRIEF SUMMARY OF THE INVENTION 

It is therefore one object of this invention to provide a 
method for capturing, transmitting, storing, and retrieving 
documents created in or converted to the electronic image 
domain which overcomes the obstacles imposed by the 
inherent speed limitations of the digital domain and the low 
resolution and perceived imprecision of the analog domain, 
yet which permits subsequent use of digital interfaces for 
interactive document processing. 

It is a related object of this uivention to design the above 
method such that it utilizes hybrid signal processing to 
accomplish the dissection of an electronic image into a 
progressively increasing number of segments and the a 
posteriori^ reconstruction of a complex electronic image or 
a multi-image document from those segments. 

' Literally "after the feet" or "deduced from experience;" implying going 
from effect to cause, or from fundamental components to complex stnictore, 
by reversing a previous process or chain of logical operations. 

It is thus another object of this invention to design the 
above method such that it permits the unrestricted decon- 
struction or disassembly of a source document in order to 
optimize the informational content and processing time 
needed to accomplish a particular document processing 
operation using specified equipment to achieve a desired end 
result, and further permits the subsequent seaming together 
of a virtually unlimited number of related segments (or 
umelated image memory maps) to reconstimte or create 
documents having extensive physical dimensions. 

It is accordingly an additional object of this invention to 
design the above method such that it employs both digital 
and analog signal processing techniques at predetermined 
points to achieve the particular advantages associated with 
the respective signal processing domain, and minimize or 
eliminate the adverse characteristics of the converse domain. 

It is an object of this invention to provide a system for 
processing documents created in or converted to the elec- 
tronic image domain which treats the electronic image in a 
"fundamental" form independent of any transformation or 
manipulation, and which is therefore universally compatible 
with any existing or later established standard or format for 
digital input, output, transformation (including compres- 
sion), presentation, or representation. 

It is an associated object of this invention to facilitate the 
transparent integration of such a system into current inter- 
active document processing environments. 

It is yet another object of this invention to design the 
above system so as to introduce virtually imlimited scalabil- 
ity for capture, storage and retrieval, and transmission 
capacities to relatively large document processing systems 
and databases, particularly for networks in which the 
fileserver is a mainframe computer. 

It is a unique object of this invention to design a system 
for the capture of electronic images from tangible source 
documents, and the subsequent transmission, storage, and 
retrieval of those electronic images, which permits complete 
decentralization of document processing operations among 
remote networks, and permits remote operators to transmit, 
store, retrieve, and receive documents with the same capa- 
bilities and access available to a central document process- 
ing or coordination facility. 
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It is a related object of this invention to increase .the 
modularity of a document processing environment by link- 
ing storage and retrieval functions performed by peripheral 
devices with^the processors responsible for document trans- 
formation and presentation, thus enhancing the operational 5 
capabilities of the entire system compared with existing 
technologies, and to further increase the modularity of the 
system by linking components by both conventional and 
high speed networks. 

It is a further object of this invention to design the above lO 
system such that it may be constructed using currently 
available recording technologies and established frame for- 
mats, as well as subsequently developed technologies and 
formats for recording electronic images including but not 
limited to those for optical laser disc recorders utilizing is 
blue-green lasers, greater bit depths, and multiple substrates. 

It is a related object of this invention to design embodi- 
ments of the above system incorporating time-based conec- 
tions to allow the invisible seaming of segments in an image 
and correcting time-based errors that are introduced by 20 
conventional recording devices such as analog optical laser 
disc recorders, as well as incorporating calibration controls 
that permit *'on the fly" adjustment of grayscale levels 
between lines or segments, maintain grayscale consistency 
and precision throughout all frames, and permit selective 25 
viewing enhancement by displaying a range of grayscale 
levels corresponding to a desired amplitude examining area 
within the image at the optimal base level for an existing 
display device and for a particular application (similar to the 
effects created by adjusting "brightness" and "contrast" to 30 
optimize a grayscale range within a displayed image.) 

It is a specific object of this invention to design several 
embodiments of the above system for capturing, storing, and 
retrieving electronic images from within an interactive docu- 
ment processing network which permits electronic images 35 
corresponding to US letter-sized pages (or smaller) to be 
captured and stored in at least 8-bit grayscale at batch 
processing rates of 5 to 30 source documents per second or 
greater (depending upon the image size and informational 
content), 'and transmitted at rates of 18,000 to 108.000 40 
documents per hour or greater. 

It is a related object of this invention to design the above 
system such that it may be utilized to establish separate high 
speed network and facsimile transmission protocols for 
transmitting documents at equivalent speeds, as well as 45 
integrated with existing networks and communications sys- 
tems in a manner that would increase the transmission rates 
which may be achieved using conventional transmission 
pathways. 

It is an object of this invention to design the above system 50 
such that transmission operations will be compatible with 
traditionally non-document based communications systems, 
including such alternatives as multichaimel RF television 
transmission, broadcast cable television transmissions, 
microwave and satellite transmission, and be compatible 55 
with conventional communications-based security tech- 
niques such as line scramblers and encryption algorithms. 

It is a distinct object of this invention to design the above 
system such that it may be immediately implemented in 
existing document processing etivironments (such as finan- 60 
cial institutions, medical centers, universities, businesses, 
and research facilities) using conventional interactive docu- 
ment processing hardware (personal computers, networks, 
and mainframes), and commercially available equipment for 
image capture and mass storage which adhere to existing 65 
uniform standards and formats. 

It is therefore a related object of this invention to design 



embodiments of the system such that they may be interfaced 
with conventional personal computers and local area net- 
work systems operating with existing commercial capabili- 
ties, one representative example being a personal computer 
having an ISA bus, an Intel 486 processor, a clock speed on 
the order of 33-66 Mhz, 70 nanosecond RAM. and inter- 
faced with a conventional Novell or Ethernet network. 

It is another object of this invention to design embodi- 
ments of the system wherein each image may be stored on 
a unit of storage medium and addressed by a volume and 
frame number, and wherein a field may be added to estab- 
lished databases in order to correlate batch-processed docu- 
ments with their original sotnce docimients or transactions, 
or conversely a separate database may be utilized as a 
"look-up table" to identify or retrieve the volume and frame 
number of the stored image corresponding to a specific 
source document or. transaction. 

It has been proposed that breakthroughs in the operational 
capabilities of five technological areas will be necessary to 
convert the majority of existing business and technical 
document processing environments from "primarily tan- 
gible" to "primarily electronic," thus achieving an effec- 
tively "paperiess" document processing environment (ine- 
spective of source document form.) Hiose basic 
technologies are storage/retrieval servers comprising file 
storage and indexing capabilities, transformation servers to 
permit complete or uruversal compatibility of document . 
formats, communication servers that provide transmission 
interfaces for both networks and remote transmissions (i.e., 
satellite, fiber optic, telephone), recognition servers capable 
of capturing and preserving the fundamental information 
from tangible documents, and document managers to track 
the location, flow, and transformation of documents. 

It is tiierefore an object of this invention to design the 
method and system of this invention such that it achieves the 
goal of providing suf&cient hardware capabilities in each of 
the five basic technological areas that would permit a 
primarily electronic or "paperless" document processing 
environment for business and technical applications. 

It is a related object of this invention that the system 
facilitate the retention of electronic images corresponding to 
the "original" tangible source documents which contain and 
preserve significandy'more usable information than can be 
appreciated or recognized by human visual perception of the 
source document, and which further may provide an audit 
trail and security capabilities to ensure and verify the 
autiienticity of the fundamental electronic image. 

It is a further object of this invention to design the 
hardware components necessary to establish an electronic 
publishing and archive system (such a system having pre- 
viously been postulated to maintain every "original" docu- 
ment in storage on only one of a plurality of file servers that 
may each be accessed independentiy by any operator within 
the system, and in which distinct documents may be con- 
nected to one another using hypertext pointers) within 
existing time and expense parameters for users of conven- 
tional business and technical informational databases. 

Briefly described, the invention comprises a method and 
apparatus for the high speed conversion of tangible source 
documents to the electronic image domain, and the subse- 
quent transmission or storage and retrieval of those elec- 
tronic images, utilizing hybrid (analog and digital) signal 
processing. The system employs a higher bandwidth analog 
signal for image capture and lower bandwidth analog signal 
for transmission or storage and retrieval, with an intervening 
digital memory buffer utilized to construct a bitmap of the 
image to facilitate various dissection and seaming schemes 
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which optimize image content and processing time depend- 
ing upon the size of the source document, the informational 
content or "resolution" necessary for the particular applica- 
tion, and the particular storage mediurh or transmission 
pathway being considered. 5 

The system is designed around a conventional bus stmc- 
ture, and the memory buffer may also serves as a transparent 
and imiversal junction with conventional interactive docu- 
ment processing systems including personal computers, 
networks, transmission systems, and various types of periph- lo 
eral input/output devices. The system also processes elec- 
tronic images in a maimer that provides complete compat- 
ibility with formats and standards for digital storage, image 
transformation and compression, local area networks, and 
communications systems. 15 

In a representative embodiment, the electronic image 
corresponding to a tangible source document such as an 
8Vi"xll" page is captured using a camera producing an 
analog output signal with conventional raster synchroniza- 
tion. The vertical and horizontal synchronization pulses are 20 
stripped from the analog signal, which is then digitized in 
8-bit grayscale and multiplexed to the memory buffer where 
the image exists as a digital array or bitmap that may be 
divided irito a plurality of segments. Tbe digital content is 
read from the memory buffer according to this dissection 25 
scheme, converted to an analog signal, and control signals 
are added. The control signals include horizontal and verti- 
cal synchronization pulses (and interval blanking), a pilot 
signal to maintain alignment along the seams between 
adjacent segments of the electronic image when it is recon- 30 
stimted, and calibration pulses to permit instantaneous 
adjustment of the base grayscale level for each line of the 
electronic image to maintain a consistent grayscale level 
The resultant analog signal is stored on a randomly acces- 
sible storage medium such as a conventional analog optical 35 
laser disk recorder (LDR) as one or more frames (each frame 
corresponding to a segment of the electronic image), or the 
resultant signal may alternately be transmitted to a remote 
location and reassembled, or up converted and displayed on 
a conventional monitor. 40 

The primary advantages of the technology disclosed 
herein are therefore speed, increased qualitative integrity, 
enhanced image content, and universality or translatability 
of image content. The most compelling rationale for appli- 
cation of this modality in present business or commercial 45 
envirorunents is the increased speed in document process- 
ing. There is a lesser demand to increase or enhance image 
content or qualitative integrity, however the inherent 
increase in stored image content that may be achieved 
without increasing storage densities or inti:oducing undue 50 
expense or complexity to the overall system does have the 
effect of making this modality optimal for processing certain 
tangible documents as electronic images (such as written 
records of financial transactions, medical imaging, and dam- 
aged or imreadable originals) where preservation and repro- 55 
dudbility of all fundamental information in the tangible 
document and the corresponding certification of security and 
authenticity permit electronic document processing to be 
applied where it was previously too costly, complex, vola- 
tile, or susceptible to challenges of unfitness or impropriety. 60 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a diagrammatic depiction of an interactive 
document processing network utilizing the electronic docu- 65 
ment transmission, storage, and retrieval system of this 
invention for the capture, transmission, storage, and retrieval 
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of electronic image documents; 

FIG. 2 is a schematic diagram showing the components of 
an embodiment of the electronic document Uransmission, 
storage, and retrieval system of this invention particularly 
adapted for the capture, transmission, storage, and retrieval 
of electronic image documents; 

FIG. 3 is a diagrammatic depiction of an 8V4"xH" tan- 
gible source document oriented in the upright or vertical 
position; 

FIG. 4 is a diagrammatic depiction of the source docu- 
ment of FIG. 3 oriented in the sideways or horizontal 
position, with a phantom line showing the source document 
divided corresponding to two horizontally adjacent image 
segments, and further showing two sub-sections correspond- 
ing to the front and back of a conventional negotiable 
instrument (check) superimposed over the right segment of 
the source document; 

FIG. 5 is a diagrammatic depiction of the source docu- 
ment of FIG. 3 oriented in Ihe sideways or horizontal 
position showing the orientation of typed Unes; 

HG. 6 is a diagrammatic depiction of a source document 
such as a C-size drawing, with phantom lines showing the 
source document divided into four horizontally and verti- 
cally adjacent regions corresponding approximately in size 
to the source document of HG. 3; 

FIG. 7 is a diagrammatic depiction of a source document 
(such as a seismic tracing) corresponding approximately in 
size to several of tiie source documents of HG. 3; 

FIG. 8 is a diagram showing three grayscale gradients in 
which 8A is a 4-bit grayscale gradient printed at 300 dpi 
showing 12 gray levels including black and white, SB is 
8-bit grayscale printed at 300 dpi resolution showing the 
dithering pattern appearing as dots of progressively increas- 
ing diameter from top to bottom, and 8C is the same 8-bit 
grayscale printed at 3360 dpi with a 150 line screen showing 
the same dithering pattern with approximately 58 gray levels 
appearing as horizontal banding; 

FIG. 9 is a diagranunatic depiction of a single capture 
device utilized to capture an image associated with the 
source document of FIGS. 4 or 5, with a phantom line 
showing the source document divided corresponding to two 
horizontally adjacent image segments; 

FIG. 10 is a diagrammatic depiction of a plurality of 
capture devices utilized to capture an image associated with 
a source document greater in size than the source document 
of FIG. 3, with a phantom line showing the source document 
divided corresponding to two adjacent image segments; 

FIG. 11 is a diagranunatic depiction of a plurality of 
capture devices utilized to capture an image associated with 
both the front and back faces of a source document such as 
the negotiable instrument (check) shown in HG. 4; 

HG. 12 is a diagrammatic depiction of a plurality of linear 
array type capture devices configured in an array utilized to 
capture an image associated with a source document greater 
in size than the source document of HG. 3, with phantom 
lines showing the source document divided corresponding to 
an equal plurality of adjacent image segments; 

HG. 13 is a diagrammatic depiction of a line array type 
capture device utilized to capture an image associated with 
a source document, with phantom lines showing the source 
document divided corresponding to an image segment; 

HGS. 14(a-i) are waveform diagrams in which 14a 
shows synchronization pulses and blanking intervals on 
opposing sides of a segment of analog signal; HG. 14b 
shows the analog signal of HG. 14a after digitization; HG. 
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14c shows the digital signal split and expanded to form two 
digital segments; I4d shows the two digital segments of 
FIG. 14c converted to analog signals; 14tf shows two posi- 
tive-going calibration pulses; 14/ shows a negative-going 
synchronization pulse; 14g shows a low level pilot signal; 5 
14h shows a composite wavefonn of the two active signals 
of FIG. Ud summed with the signals of FIGS. 14c-14^ 
overlaid over the sum of the signals of FIGS. 14e-14^; and 
I4i is a square wave corresponding to the clock speed 
utilized to . reconstruct the electronic image represented by 10 
the analog signal of FIG. 14a; 

FIG, 15 is a diagram showing the arrangement of seg- 
ments and firames in Mode CI (line delay) overlying a raster 
image composed of horizontal scanning lines; 

FIG. 16 is a diagram showing the arrangement of seg- 
ments and frames in Mode C2 (field delay) overlying a raster 
image composed of horizontal scanning lines; 

FIG. 17 is a diagram showing the arrangement of seg- 
ments and frames in Mode C3 (one frame delay) overlying 
a raster image composed of horizontal scarming lines; and 

FIG. 18 is a diagram showing the arrangement of seg- 
ments and frames in Mode C4 (two frame delay) overlying 
a raster image composed of horizontal scanning lines. 

25 

DESCRIPTION OF 'IHE PREFERRED 
EMBODIMElSfTS 

Hie electronic document transmission, storage, and 
retrieval system of this invention is represented in FIGS. 
1-14 and referenced generally therein by the numeral 10. 

Referring particularly to FIG. 1, an interactive document 
processing environment is shown which includes a low 
speed local area network (LAN) 12 such as a Novell or 
Ethernet network or a peer-to-peer network system linking 
a plurality of nodes which may be occupied by personal 
computers 14, terminals, or workstations. The network 12 is 
optionally linked to a mainframe 16 or file server along a 
conventional digital communications pathway. Each com- 
puter 14 on the network 12 is linked to various peripheral 
devices including a digital document input device 18 such as 
a document scanner, an output or representation device 20 
such as a laser printer or film recorder, and a transmission 
interface 22 such as a modem or facsimile machine. Each 
compute 14 or terminal preferably has various input and 
output devices for the operator including a keyboard, mouse, 
trackball, or pressure sensitive tablet, storage mediums such 
as a magnetic or optical disk, tape cartridge, or compact disc 
drive, and at least one presentation system such as a raster 
display or LCD projection screen associated therewith-. 

Each of the computers 14 in the network 12 are simulta- 
neously connected to a separate high speed network 28 
which includes an interface module (not shown) installed in 
or linked to each of the computers 14. The interface module 
will preferably have the capability for at least retrieval and 55 
transmission of documents utilizing the high speed network 
28, and may optionally include storage capability. The 
network 12 may also be coimected to a remote network 24 
through a conventional transmission pathway 26 such as 
telephone lines or a dedicated cable system, and a "high- go 
speed" transmission pathway 26' capable of transmitting 
electronic documents at rates equivalent to the faster oper- 
ating speeds of the high speed network 28 of the system 10. 

The high speed network 28 is linked to an electronic 
image server 30, which is in turn connected to a capture 65 
device 32 and a mass storage device 34. The mass storage 
device 34 is cormected to the electronic image server 30 by 
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both a conventional digital communication pathway and a 
high speed pathway. The mainframe 16 is also connected to 
the electronic image server 30 along a digital communica- 
tions pathway. 

The electronic image server 30 may be self-contained 
within its own housing and include the necessary andllaiy 
components such as a power supply, operator interface, and 
the like, or may be fabricated on a separate interface module 
mounted in a host processor such as a personal computer or 
workstation. It is understood that each computer 14 in the 
networic 12 that is linked to the high speed network 28 will 
also include a separate EIS interface 30 which has some or 
all of the processing capabilities of the electronic image 
server 30. Thus, for purposes of clarity, the term electronic 
image server 30 will be used to refer to the separate unit as 
shown in FIG. 1 that is used primarily for capture and batch 
processing operations, and the term EIS interface 30 will be 
used to refer to a computer-based interface associated with 
each computer 14 or workstation. 

Each EIS interface 30 may include some or all of the 
components necessary for storage, retrieval, transmission, or 
presentation depending on the number and extent of docu- 
ment processing operations that will be performed at that 
station or node. As such, the EIS interface 30 is hereafter 
assumed to have the full functional capabilities of, and be 
functionally indistinguishable from, the separate electronic 
image server 30 unless otherwise noted, and the schematic 
shown in FIG. 2 will therefore delineate the relationships of 
both the separate electronic image server 30 and the com- 
puter-based EIS interface 30 as being mounted on a host 
processor such as a personal computer or workstation. 

In normal operations, images from tangible so\irce docu- 
ments will be captured using the capture device 32 and 
stored on the mass storage device 34. A volume and fi*ame 
number for each image will be written to a database in the 
mainframe 16 using the conventional digital communica- 
tions pathway. An operator at a computer 14 on the network 
12 will issue a retrieval command to the mainframe 16 for 
a particular document image, and the mainframe 16 will 
access the database to determine the correct volume and 
frame numbers for the image. The mainframe 16 will issue 
an instruction through the digital communication pathway 
(and electronic image server 30) to the mass storage device 
34, which will retrieve the appropriate number of frames 
from the storage medium in the mass storage device 34 and 
transmit those frame over the high speed network 28 to the 
requesting computer 14. The EIS interface 30 within the 
receiving computer 14 will reconstmct the electronic image 
for the operator to view, and the operator may process that 
electronic image as desired. Alternately, an image may be 
retrieved from the mass storage device 34, reconstituted in 
the electronic image server 30, and transmitted along the 
digital network 12 through the mainframe 16 and to the 
requesting computer 14. 

If the operator woridng at one of the computers 14 or 
workstations makes a transformation to the electronic image 
and wishes to store that derivative image or a newly created 
electronic image, there are two altemativcs. Fint, the opera- 
tor could issue a store command which would cause the 
electronic image to be transmitted over the high speed 
network 28 to the mass storage device 34, with a corre- 
sponding instruction being sent to the mainframe 16 over the 
conventional conanmnications pathway which would update 
the database with any information concerning the processing 
history of the electronic image, and also initiate the appro- 
priate subroutine in the mainframe 16 to instruct the mass 
storage device 34 to receive and store the correct number of 
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frames, and obtain the volume and frame numbers from the 
mass storage device 34 to be placed in the database. Alter- 
nately, the operator could send the electronic image in digital 
form through the mainframe 16 to the electronic image 
server 30 using the conventional digital communication 5 
pathways, where the image would be converted to frames 
and stored on the mass storage device 34 with the database 
in the mainframe 16 being similarly updated. 

The remote network 24 will ordinarily be associated with 
a separate mainframe 16' or fileserver, and a separate high 
speed network 28' with a separate mass storage device 34' as 
well as at least one separate electronic image server (not 
shown) and the associated peripherals and components. If an 
operator wishes to transmit an electronic image to the 
remote network 24, the electronic image 24 may be trans- 15 
mitted over the high speed transmission pathway 26' to a 
high speed network 28' linking the nodes of the remote 
network 24, where it may be saved on a separate mass 
storage device 34' and a database in the remote mainframe 
16* will perform the necessary cataloging and indexing 20 
functions. As discussed in greater detail below, it may be 
appreciated that images may be downloaded in batches 
directly from one mass storage device 34 to another 34' 
using the associated high speed networks 28, 28' and the 
high speed transmission pathway 26', with the correspond- ^ 
ing database information being transmitted from one main- 
frame 16 to the other mamframe 16' along the conventional 
digital transmission pathway 26. 

Hie capture device 32 could be any conventional high 
speed image capture device, although as discussed herein 
the capture device 32 and electronic image server 30 are 
preferably designed, structured, and tuned to match the 
particular document processing applications for which the 
high speed network 28 will be utilized. 

Referring particularly to FIG. 2, the various components 
associated with the EIS interface 30 and interconnections 
between the EIS interface 30 and the host processor are 
shown. 

The host processor indudes a communications pathway 40 
such as a bus structure 36 which may be of any conventional 
type defining or defined by the architecture of the 1 host 
processor. Although an external bus of the type utilized for 
real time data acquisition and control systems could be 
utilized, it has proven suitable to design the embodiments 45 
discussed herein around the main bus structure 36 of the host 
processor. It is understood that the bus structure 36 of most 
personal computers and work stations is hierarchical, includ- 
ing a backbone or system bus, and one or more subsystems 
such as an instmction exchange bus, a data exchange bus, an 50 
arbitration bus to allocate access among competing devices, 
and an interrupt bus to accept attention request signals from 
devices. The bus architecture also dictates such features as 
the motherboard, slot, and card dimensions, grouping and 
cooling of components, power distribution, and cormector 55 
configurations and arrangements. 

There are currentiy believed to be twelve accepted full 
standards for bus structures 36 recognized by the Institute of 
Electrical and Electronic Engineers (IEEE), and over four- 
teen proposed standards. There are also many bus standards 60 
that have been designed by special interest groups with the 
cooperation of other organizations to satisfy particular appli- 
cations, and there are several proprietary bus standards tiiat 
remain unpublicized. Adapters may also be attached to the 
system bus to access specialized interface buses such as the 65 
Small Computer System Interface (SCSI) for disk drives, 
scanners, and most other input/output devices, and the 
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General Purpose Instrumentation Bus (GPIB) for measure- 
ment and control devices. 

Some of the bus structures contemplated for use with the 
electronic document transmission, storage, and retrieval 
system 10 include PCBus (IBM PC class), ISA (IBM AT 
class), MicroChannel (IBM 32-bit OS/2 class), STD Bus, 
EISA (Compaq 32-bit), S-100 and SBus (Sun Microsys- 
tems), VME (Motorola), open Unibus and QBus (Digital/ 
DEC), and MultiBus and MultiBus II (Intel). It may be 
readily appreciated that since any component comiected to a 
personal computer bus stmcture 36 must operate with the 
host processor and all other peripheral devices, the bus 
structure 36 of the host processor will dictate many of the 
design characteristics of the EIS interface 30 and related 
components. The embodiments subsequently discussed are 
suited for applications utilizing the ISA bus on IBM com- 
patible computer systems incorporating Intel 386 or 486 
processors. 

The bus structure 36 of the host processor is preferably 
equipped for connection to a digital capture device 38 such 
as a document scanner through a peripheral interface 40, a 
local area network 12 or peer-to-peer network through a 
separate network interface 42, an on board or external device 
capable of image compression 44 using any conventional 
compression standard such as JPEG, and an output device 46 
such as a laser printer of film recorded connected through a 
serial or parallel output interface 48. 

The bus structure 36 may of course be connected to many 
other types of peripheral devices, however these are the 
basic devices utilized in the capture, representation, and 
transmission of electronic image documents over existing 
networks 12 adapted for mteractive document processing. 

The operator will have direct command access to both the 
bus structure 36 and the interface for the EIS interface 30 
through a keyboard 50 or other devices such as a mouse, 
trackball, tablet, or the like, as well as through any input/ 
output and peripheral devices connected to the bus structure 
36. 

High speed document capture is accomplished using a 
capture device 52 which provides an analog output signal 
timed with synchronization and blanking pulses correspond- 
ing to individual frames following the normal . fast-scan 
television system standards of 30 frames per second with 
two interlaced fields or switchable to one progressively 
scanned field per frame, as modified according to the pro- 
visions discussed subsequentiy hereia The capture device 
52 may be any conventional chargc-couplcd device (CCD) 
or Vidicon tube type camera, but preferably has a progres- 
sive or sequential scan, horizontally and vertically synchro- 
nized analog output signal that is switchable between con- 
forming to the fast-scan television system standards 
producing 30 frames per second and slower vertical sweep 
rates as required by high resolution modes of operation. An 
alternative would be to have two cameras to switch betwecru 
It may be appreciated that conventional CCD cameras 
having from less than 512x512 to greater than 2048x2048 
pixel arrays will be suitable for many applications employ- 
ing the modality disclosed herein, and that non-square, 
larger and small arrays, and linear array cameras or scanners 
may be equally suited for some applications. 

The analog output signal .from the capture device is 
processed through an analog-to-digital converter 54 timed 
using an input sync-gcncrator 56, with the system being 
driven by a 124 Mhz clock speed oscillator (crystal) with 
input to die analog-to-digital converter 54 divided by four to 
achieve a 3 1 Mhz conversion clock speed. Since the result- 
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ant output signal is still a time-division multiplexed signal, 
a 3 1 Mhz to 3.9 Mhz demultiplexer 58 may be used to strip 
the timing and synchronization pulses from the output signal 
and distribute the remaining image signal to an input 
memory buffer 60 with a 3.9 Mhz clock rate that is com- 5 
patible with a broader spectrum of suitable memoiy devices 
that are currently commercially available. 

It may be appreciated that in the embodiments discussed 
an 8-bit analog-to-digital converter 54 is used to obtain 
digital content corresponding to an 8-bit grayscale bitmap* jq 
however the continuous analog signal output from the 
capture device 32 would permit any the use of an analog- 
to-digital converter 54 that would produce any selected bit 
level in either grayscale or color depending upon the par- 
ticular application being considered and informational con- 
tent desired. 

The image input memory 60 is preferably an SRAM 
semiconductor memory. It should be noted for purposes of 
this discussion that classification of semiconductor memory 
usually includes dynamic and static random access memory 
(DRAM and SRAM), read-only memory (ROM), as well as 
programmable ROMs (PROM), erasable ROMs (EPROM, 
EEPROM). nonvolatfle RAM (NVRAM), and flash- 
memory such as charge-coupled devices (CCD) and mag- 
netic bubble memory. Each type of memory may be dassi- 
fied as dynamic or static, read/write or read-only, random or 
serial access, volatile or nonvolatile, and erasable using 
magnetic or ultraviolet radiation. Because a CCD sensor 
chip is properly categorized as a type of serial access 
semiconductor memory, it is important to distinguish ^ 
between the bitmap or digital array corresponding to an 
electronic image formed on the CCD sensor chip of a linear 
imaging device (LID) such as a line-array scanner or an area 
imaging device (AID) camera, and the digital army or 
bitmap associated with the corresponding image once the 
digital information has been read from the capture device 32. 
converted and multiplexed for analog television output. , 
interlaced if necessary for video, and subsequently digitized 
by a ftame-^bber or similar device and deposited in active 
memory which may or may not be user-interactive memory. 
Consequently, the memory buffer 60 connotes digital semi- 
conductor memoiy other than the image sensor itself which 
permits interactive addressing and manipulation of pixels or 
swapping of information into memory fipom which pixels 
may be interactively addressed and manipulate, and which 
may be configured as an interactive junction with a bus 
structure 36 or similar interfaces. 

The digital array associated with a particular electronic 
image may be stored to or retrieved from a conventional 
digital storage device 62 such as a magnetic or optical disk 50 
drive, or transmitted to or through any conventional periph- 
eral, through the bus structure 36. It should also be noted that 
a previously stored electronic image document may be 
recaptured using the analog capture pathway by substituting 
an alternate image input source 64 such as a write-once/ 53 
read-many-times (WORM) optical laser disc player (LDP) 
connected through a mechanical or electronic switch 66 to 
the input channel of the analog-to-digital converter 54. A 
Sony Model LDR-50(X)A optical disc recordei/player (EIA 
Standard) has proven suitable. 60 

The switch 66 also provides a suitable junction for linking 
the input of the analog-to-digital converter 54 to the high 
speed network 28, and it may therefore be appreciated that 
an electronic switching device such as a latch controllable 
by the EIS interface 30 or the mainframe 16 is desired for 65 
switching between the capture device 52, laser disc player 
64, and high speed network 28 so that electronic images 
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transmitted on the high speed network 28 will not be 
obstructed or delayed awaiting manual switching. A manual 
switch 66 would preferably have a default setting to the high 
speed network 28, with user-selected switching to the cap- 
ture device 52 or laser disc player 64 intended only for 
intermittent operation. 

At this point on the image processing pathway, the 
electronic image document exists as a digital array or bitmap 
in active memory. Hie digital array or bitmap may not have 
the same physical X-Y coordinate array as was present on a 
CCD sensor chip. In fact, for the applications subsequently 
discussed, it has proven preferable to divide or section the 
image into blocks or segments, and to further deposit 
portions of each block or segment in distinct banks of 
memory. However, each pixel remains mapped to a specific 
memory location by bank, row, column, and phase by a 
matrix or formula in the memory control module 68, and a 
set digital array or bitmap is preserved and recognized even 
though it may require calculation to conelate a pixel's 
location in memory to its position on a raster line or LCD 
display. 

The electronic image document may then be stored on an 
analog storage device 70 such as an analog WORM optical 
laser disk recorder (LDR) by dumping the digital array from . 
memoiy 60 as a conventional 4 Mhz clock speed digital 
transmission signal and processing that signal through the 
first step of a sequential frequency converter 72 (up con- 
verter) to achieve a 16 Mhz bandwidth, an 8-bit digital-to- 
analog converter 74. and multiplexer 76 producing a 6 Mhz 
bandwidth output and the appropriate timing and synchro- 
nization pulses expected by the storage device 70. 

A switching device 66 may also be utUized to provide a 
suitable junction for linking the output of the multiplexer 76 
to the high speed network 28 for transmission of electronic 
images, and it may be appreciated that conversion of the 
signal to the appropriate bandwidth for the high speed 
transmission pathway may require that the up converter 72 
or the multiplexer 76 be instructed by the EIS interface 30 
to perform the appropriate signal conversion steps to match 
the bandwidth of the analog signal to the corresponding 
transmission pathway. 

The 16 Mhz clock rate signal from the first step of the 
frequency converter 72 may also be processed in a second 
step to achieve a 124 MHz clock rate signal, converted to 
analog using a digital-to-analog converter 78, and presented 
on a conventional raster display 80, preferably such as a high 
resolution computer monitor. In some applications subse- 
quently discussed, separate input and output memories may 
be utilized, and the output memory may be read at a speed 
directiy compatible with monitor inputs. The image may 
also be presented on a conventional LCD display screen 82 
or projector, although the necessary signal form may need to 
be created as required by the LCD display 82. 

Since each electroruc image document is stored on the 
storage device 70 in the form of one or more frames 
(depending upon the operational mode being used and the 
size of the image), a database 84 is utilized to record an 
accession number, index, or address for the initial frame and 
size (or for each of the separate frames) associated with a 
specific document to permit cataloging and selective 
retrieval of any document stored on that medium. The 
database 84 may be located in the mainframe 16 or 
fileservcr, a separate CPU associated with a remote EIS 
interface 30 device, or be resident in the computer housing 
the bus structure 36. In many applications involving high 
speed batch processing on the order of several million 
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Source Document Orientation 

The embodiments of the system 10 discussed herein have 
been specifically tailored to optimize the processes of cap- 
turing and storing electronic images from tangible source 
documents that are especially common to many financial 
and business transactions conducted in the United States, or 
for specialized applications that are considered document 
intensive. Those tangible source documents include paper 
documents, as well as documents previously fixed in medi> 
urns such as microfilm and microfiche. The following dis- 
cussion of several paper-based source documents provides a 
suflcient basis for describing the particular modes of opera- 
tion associated with the preferred embodiments. 

Referring particularly to FIGS. 3-7, several of these 
source documents are shown. FIG. 3 is a diagrammatic 
depiction of a basic US letter-size page 90 generally 
approximating the horizontal and vertical dimensions or 
aspect ratio of a standard 8Vi"xir* sheet of paper (and 
similarly approximating an A-4 size page) in a vertical 
orientation. FIG. 4 is a diagrammatic depiction of the same 
page 90 shown in FIG. 3 in a horizontal orientation. Hie 
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tangible source documents per day (such as checks or 
similar negotiable instruments processed by a financial 
institution, for example) each stored image will be identified 
by a volume number corresponding to the serial number of 
the specific disc or recording medium, and a firame number. 
A single frame number will be satisfactory when all batch- 
processed images are stored using the same quantity of 
frames for each image, and the combination of the volume 
and frame numbers may be treated together as a single 
index, address, or accession number. The database 84 may 
be an already-existing database used for processing trans- 
actions involving the source documents to which one or two 
fields are appended corresponding to the volume and frame 
uTimbers for the stored image, or may be a separate database 
84 which provides a "look-up table" relating an identifying 
characteristic of the source document (such as the computer- 
readable MICR code on negotiable instruments or checks) 
with the corresponding serial and frame numbers of the 
stored image. 

Because the speeds at which tangible documents may be 
captured exceeds the rate at which an operator can manually 
position the document, it is anticipated that automatic docu- 
ment feeders (not shown) will be utilized particularly with 
paper, film, and microfiche documents. Consequentiy, an 
infeed control interface 86 must be connected to the EIS 
interface 30 so that the EIS interface 30 can receive ready- 
state acknowledgements from the automatic . document 
feeder and step or advance commands can be sent to the 
automatic document feeder, and a corresponding indexing or 
accession control interface 88 would be connected between 
the EIS interface 30 and the storage device 70 to correlate 
the volume serial numbers and frame numbers with the 
appropriate images that are stored, and provide sequencing 
and control signals to the storage device 70. A manual infeed 
control interface 86 would be used when documents are 
positioned manually by the operator. 

A separate storage device 70' such as a second analog 
WORM optical laser disc recorder (LDR) may be utilized at 
a remote location for batch downloading or backup of 
images stored on the first storage device 70, with the 
corresponding indexing or cataloging information from the 
database 84 being supplied to a separate database that is 
connected to the second storage device 70*. 
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page 90 is divided along an imaginary vertical centerline 92 
into two half-page segments 94, 96 having dimensions of 
approximately 8H"x5V4" each. Two blocks 98 each repre- 
senting the front or back face of a standard check (negotiable 
instrument) having dimensions of approximately 2Wx6" 
are superimposed on the right half 96 of the page 90. FIG. 
5 is a diagranmiatic depiction of the page 90 of FIG. 3 
showing the normal orientation of type written lines 100 of 
text on that page 90. 

HG. 6 is a diagrammatic depiction of a drawing sheet 102 
having dimensions greater than twice a basic page 90, with 
horizontal and vertical phantom lines 104 showing the 
drawing sheet 102 divided into four equal quadrants or 
segments along both horizontal and vertical seam lines. 

HG. 7 is a diagrammatic depiction of a section of a 
large-scale document 106 such as a geophysical survey. Hie 
section 106 has dimensions on the order of 36"x48'* and may 
correspond to about 4 miles of survey data with 50 samples 
per mile along the horizontal axis, with the entire document 
encompassing 70 or more miles of survey data and having 
a total length of many feet. 



Capture Device Arrays 

Referring particularly to FIGS. 9-13, it may be seen that 
the processes of capturing and storing electronic images 
from the types of source documents 90, 94, 98, 102, 106 
discussed above may be accomplished in several maimers 
depending upon the corresponding size of the source docu- 
ment 90, 94, 98, 102, 106 and the degree of informational 
content desired to be captured and stored. 

HG. 9 is a diagrammatic depiction of a single capture 
device 32 such as an area array CCD or Vidicon tube type 
camera disposed a predetermined distance or height above a 
source document such as a basic page 90 oriented horizon- 
tally, with a phantom line 92 identifying an imaginary 
centerline 92 dividing the basic page 90 into two half-page 
(one frame) segments A, B. HG. 10 is a diagrammatic 
depiction of a pair of capture devices 32 disposed above a 
source document 108 such as a basic page 90 oriented 
vertically or two-page drawing sheet 102 oriented horizon- 
tally, with a phantom line 110 identifying an imaginary 
centedine or seam line dividing the source document 108 
into two segments A, B (of two frames each). 

HG. 11 is a diagrammatic depiction of a pair of capmre 
devices 32 disposed above and below a source document 
112 such as a check, with one capture device 32 capturing 
a segment A of the resulting half-page electronic image 
corresponding to the front face of the check or source 
document 112, and the other capture device 32 capturing a 
segment of the resulting half-page (one frame) electronic 
image conesponding to the opposing face (not shown). 

HG. 12 is a diagranmiatic depiction of a plurality of 
capture devices 32 disposed in a two dimensional (vertical 
and horizontal) array above a large-size source document 
114, with each capture device 32 capturing a segment A-F of 
the resulting (multi-firame) electronic image which adjoin 
one another along both vertical and horizontal seam tines 
116. 

HG. 13 is a diagrammatic depiction of a capture devices 
32 of the line array type disposed above a large-size source 
document 114. with the capture device 32 sequentially 
capturing a segment A-C of the resulting (multi-frame) 
electronic image which adjoin one another dong parallel 
seam lines 116. 
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'•Mode A" and "Mode B" Operation 

Different operational modes for the electronic document 
transmission, storage, and retrieval system 10 may be 
designed to achieve particular goals or meet certain require- 
ments, limitations, or constraints associated with the docu- 
ment or the system 10. Two modes termed "Mode A" and 
'*Mode "B" arc particularly designed around document 
processing involving tangible source documents such as a 
US letter-size page 90 or smaller, 8-bit grayscale informa- 
tional content, and wherein '^normal" system resolution is 
suitable. It should be noted that "normal" resolution refers to 
an arbitrary resolution as defined by the particular system 
mode and the application of that mode to a specific operating 
environment. In other applications, normal resolution might 
refer to fractions or multiples of the "normal" resolution as 
defined by Modes A and B. It should also be noted that 
bypassing the bandwidth filters or bandwidth limiters in 
conventional analog WORM-type LDRs 70 to operate with 
the "extended" bandwidth and additional lines of image 
content results in an approximately 30% increase in the 
effective resolution or storage capacity of the storage 
medium, without affecting its storage density. These modi- 
fications are possible because the LDRs were designed 
around color broadcast specifications which differ from the 
optimal performance arrangement for document legibility. ^ 

Modes A and B are particularly adapted for a 1 -frame 30 
hz or 2-framc near 14 hz (i.e.. less than 15 hz or approxi- 
mately 70 millisecond) record time, capturing informational 
content in 8-bit grayscale from a horizontally-oriented 8V^"x 
11" tangible document. This orientation is believed prefer- 
able for documents containing typewritten characters, 
because readability of a typewritten characters requires more 
resolution in the "horizontal" direction than in the "vertical" 
direction. It may be noted that the frame rate of Mode B is 
deliberately not one half the frame rate of Mode A, since 
tube type cameras require a mini mum vertical retrace time 
and it the necessity of achieving lOlX) plus useable lines. 

Mode A refers to capturing and storing one half-page 
image of a US letter-size source document as a single frame, 
whereas Mode B refers to capturing and storing the full-page 
image of a US letter-size source document as two frames. 
Although Mode A can be considered a system bypass that 
does not require complex seaming to retrieve and reconsti- 
mte the electronic image, the same control signals including 
the pilot signal and calibration pulses as discussed below are 
applied to documents processed in Mode A, since those 
control signals are also useful for maintaining electronic 
image integrity throughout batch-processed source docu- 
ments and between different operating environments or 
platforms. 

Normal fast-scan television provides 30 frames per sec- 
ond with 525 horizontal lines swept at a 60 hz vertical rate 
and scanned at a 15,734 hz horizontal rate with a 4:3 aspect 
ratio, with two interfaced scanned fields per frame. The 55 
image is sent as an amplitude-modulated (AM) signal, while 
audio is frequency-modulated (FM). European television 
(PAL standard) utilizes 625 lines, however the aspect ration 
remains approximately 4:3. 

In Modes A and B, the capture device 52 produces a 60 
camera image of 1114 progressive (non-interlaced) lines 
refreshed every 70 milliseconds. When recorded using the 
method discussed herein on a conventional laser disc 
recorder 70 which expects a 525 line format, this will 
correspond to two images of 500 lines each with the excess 65 
lines acconmiodating the control signals as described. The 
lines of the image are therefore being scanned at 14 hz in the 
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vertical direction (approximately one quarter the normal 
sweep rate) and scanned horizontally at 15 khz. ADage-81 
camera manufactured by Dage-MTI Incorporated of Michi- 
gan City, Indiana, provides a suitable capture device capable 
of being driveti at the 14 hz vertical scan rate. It should be 
noted that more accumulated charge may be discharged as 
useable signal by lengthening the time during which a 
document is being scanned using a capture device in which 
the accumulated charge on the target is time-dependent on 
incident source light such as a Vidicon tube type camera. 
This results in enhaiiced image capture capabilities by 
effectively increasing the number of "pixels" that are con- 
tained in each scan line, although the number of scan lines 
that may be treated as a **frame" is reduced proportionately 
because each scan line is longer in duration, and the image 
therefore occupies more frames. T\ibe type cameras are 
therefore advantageous in applications where a CCD type 
camera would not provide sufiBcient resolution or fast 
enough speeds given other practical considerations, or 
where the fixed pixel resolutions of current CCD cameras 
are otherwise inadequate. While this scan rate reduction 
would normally produce a visible flicker if the images being 
captured were viewed directly from the camera output, the 
scan rate reduction is compensated for by subsequent fre- 
quency conversions that are interposed immediately prior to 
any display, projection, or other visible presentation opera- 
tion. Progressive scanning is preferred since it provides a 
KeU factor as high as 0.9 over the traditional 0.7 value for 
interlaced scanning, thus producing advantages in acquisi- 
tion and display, and since interlaced scanning results in 
greater information loss caused by inaccuracies in tracking 
adjacent sweeps and the time factor for decay occurring 
between interlaced sweeps. 

The output signal from the capture device 52 is digitized 
at 31 Mhz and stored in a 1024x2048x8 input memory 
buffer 60. The memory buffer 60 is preferably two banks of 
eight 128 kbit SRAMS with an 8 column by 9 row archi- 
tecture and a corresponding video input latch. The output 
signal is multiplexed into 8 segments which places 256 
pixels in each row of memory. Any pixel can be identified 
by bank, row, column, and phase. The configuration of the 
memory buffer 60 in Modes A and B can be made more 
efficient, however this configuration permits interchangeable 
application to other modes and is simple to construct. 

It may be appreciated from this discussion that the true 
potential of the 1024x2048 memory buffer 60 configuration 
is not necessarily being fully utilized in the embodiments 
which employ currently available state-of-the-art LDRs, 
however use of this configuration of the memory buffer 60 
provides a suitable margin to permit the use of these 
embodiments with improved LDRs currently under devel- 
opment with a minimum of adaption. As noted above, there 
is also a rationale for having separate input (capture) and 
output (display or storage) memory buffers, with each sepa- 
rate memory buffer being optimized for its intended purpose. 
Since the output from the input memory to the storage 
device is analog and the input to the output memory is also 
analog for retrieval, commonality of the digital characteris- 
tics of input and output memory is unnecessary. Separate 
memory buffers also facilitate simplicity in designing the 
interface modules to have retrieve-only capabilities, thus 
reducing the cost and complexity per node for network 
systems having a centralized high speed capture and storage 
process. 

The EIS interface 30 circuitry may be divided into three 
groups. The circuitry associated with the memory buffer 60 
is one group. The non-124 Mhz portion can be fabricated on 



07/25/2003, EAST Version: 1.04.0000 



5.502,576 



35 



36 



a first integrated circuit board using transistor-transistor 
logic (TTL), whereas the 124 Mhz portion (including the 
digital-to-analog converter 74) can be fabricated as a third 
group using emitter-coupled logic (ECL) on a separate 
integrated circuit board. The multiplexer 76 can be fabri- 5 
cated with the non-124 Mhz portion of the circuitry, or a 
remote multiplexer 76 may be connected to the non-124 
Mhz portion of the circuitry using sixty-four sets of twisted- 
pair wires. 

The resultant analog signal associated with the electronic ^0 
image may be stored on the storage device 70 in video 
format. A 525 line 60 hz vertical scan rate WORM optical 
laser disc recorder (LDR) is conventional, and a Sony Model 
LVR-5000A (EIA Standard) laser videodisc recorder/player 
has proven suitable. ^5 

Storing a 1024 line image in two standard frames pro- 
duces 26 excess lines, for which the LDR would "see" 
approximately one fourth the normal number of horizontal 
synchronization pulse durations during the vertical interval 
instead of the 21 expected, and this could affect the LDR*s 
servo lock or otherwise result in inoperability of the LDR, 
There are two immediately available options for resolving 
this situation. First, the LDR could be adapted to expect the 
narrow fine vertical intervals. However, since the use of 
unmodified standard LDRs is desired, the preferred embodi- ^ 
ment utilizes a standard vertical interval signal with a 1000 
line video image, which necessarily results in an image that 
is 24 lines short of binary "pure." 

It may be readily appreciated that a system utilizing a 625 
line 50 hz vertical scan rate LDR compatible with the PAL 
(European) television and video recoring standard may be 
preferred in situations where PAL-compatible LDRs are 
more readily accessible or in applications where the nature 
of the tangible source document makes 625 line capture the 
optimal alternative. In some applications, the increase in 
horizontal bandwidth from 4.2 Mhz to 6 Mhz may be 
compelling. Such a conversion will either decrease the 
number of images storable on a conventional optical disc by 
20% (from approximately 43,000 to 36,000) or increase the ^ 
recording time by about 20% per frame, In such an event, the 
sync generator 56 can be modified to operate corresponding 
to the 625 line 50 hz vertical scan PAL standard. However, 
it should also be remembered that the sync generator con- 
trols the drive frequency of the capture device 32 and 
supplies the timing waveforms for the input memory buffer 
60. Consequently, selection of the appropriate capture 
device 32 and optimization of memory characteristics may 
be required. 

A 525 line 60 hz vertical scan rate standard sync generator 50 
provides a composite synchronization traia After the first 
vertical synchronization pulse, the first 250 lines are read to 
the memory buffer 60. The read pauses to allow insertion of 
the second vertical interval, and then lines 251-500 are read. 
The read pauses, and the vertical interval between the first 55 
and second frames is inserted, and the second frame is read 
in the same manner as the first frame. The additional 50 line 
times are consumed by the vertical blanking intervals. At the 
same time the read starts for the beginning of the first frame, 
the LDR begins recording the output fium the digital-to- so 
analog converter 74. The image can be retrieved by revers- 
ing the storage process and reconstituting the two frames as 
a single electronic image document. 

It may be appreciated that the tangible 8^"xir' original 
image is divided into four blocks or segments to be stored on 55 
two separate frames with two blocks per frame. These 
blocks or segments can be groups of sequentially adjacent 



fines, fractions or segments of adjacent lines, alternating or 
selected interlaced fines, or any other desired portion of the 
image. An electronic image having a digital array composed 
of X fines of pixels would result in two frames each having 
X/2 lines of information, and the raster conesponding to 
those two frames would be composed of N horizontal 
scanning fines at a predetennined vertical scan rate plus a 
maximum time interval dictated by the remaining or excess 
2N-X horizontal scanning lines and the predetermined ver- 
tical scan rate which is utilized to add the raster synchroni- 
zation to the analog signal for each frame. 

The retrieved electronic image can be processed in two 
ways. It can be mapped at 3.9 Mhz onto the bus structure 36 
of the host processor and swapped to the video display RAM 
on the computer's motherboard or on a videoygraphics 
interface card, stored in digital form on magnetic medium, 
transmitted over conventional transmission pathways, or 
downloaded to an output device such as a printer. Alter- 
nately, for high resolution display the signal may be up 
converted by the frequency converter 72 to conespond to the 
appropriate dot clock equivalent for a high resolution gray 
scale or color monitor. 

••Mode C" Operation 

Modes A and B operation are considered biased in the 
horizontal direction. Mode C operation is effective where 
either "square" resolution or a vertical bias are desired. 
Mode C operation may be considered "high" resolution 
compared with Modes A or B, and provides twice the 
"normal" resolution of those modes. 

The acquisition of the signal in Mode C is the same as in 
Modes A and B, however the image is dissected into 8 blocks 
and recorded as 4 frames. The resolution of the tangible 
source document image (using the ^WxXV* example) 
becomes a 1024M X-Y pixel product (1248 pixels in the 11" 
inch direction and 1000 pixels in the 8.5" direction) while 
only requiring twice the capture and storage time. 

There arc three alternatives for operation in Mode C, In 
each case the horizontal line is split in half. The read from 
the memory buffer 60 is performed at half the write speed 
(using a 7.5 khz horizontal sweep rate controlled by the read 
clock) to effectively expand or stretch the 12 Mhz bandwidth 
signal into a 6 Mhz bandwidth signal, which causes the half 
of the horizontal line segment to expand from 32 microsec- 
onds to 64 microseconds in duration, which appears normal 
to an LDR expecting a 6 Mhz analog signal with 64 
microsecond line duration. 

Each of the three alternatives to Mode C may be under- 
stood by considering a grid overlying a 1000 line raster 
image in which each line has a left and a right half. 

In the first alternative (Mode CI in FIG. 15) the grid has 
one vertical column of eight horizontal rows. Each row of 
the grid therefore corresponds to one field of a frame. Each 
segment is therefore one vertical row consisting of 125 left 
halves alternating with 125 right halves of each line. The 
segments are stored in ascending order, SI through S8, so 
that frame Fl contains segments SI and S2 as fields 1 and 
2, frame F2 contains segments S3 and S4 as fields 1 and 2, 
and so forth. Hiis mode is the least expensive since it only 
requires a line delay, and has the advantage of providing a 
security function by "scrambling" the appearance of the 
stored images in the event an imauthorized individual 
attempts to access the medium using an incompatible EIS 
system or view the image on an interlaced monitor. 

In the second alternative (Mode C2 in FIG. 16) the grid 
has two vertical columns of four horizontal rows. Each 
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segment therefore consists of either 125 left halves or 125 
right halves of the lines. The segments are again stored in 
ascending order, SI through S8, so that frame Fl contains 
segments SI and S2 as fields 1 and 2, frame F2 contains 
segments S3 and S4 as fields 1 and 2, and so forth. This 5 
mode is the more expensive since it requires a field delay, 
and also has the advantage of providing a security function 
by scrambling the appearance of the stored images. The 
security feature of Mode C2 would produce significantly 
more flicker because of the difference between the horizon- 10 
tal sweep and vertical scan rates. 

In the third and fourth alternatives (Mode C3 and Mode 
C4 in FIGS. 17 and 18. respectively) the grid is the same as 
for Mode C2, and each segment therefore consists of either 
125 left halves or 125 right halves of the lines. In Mode C3 15 
and Mode C4, however, the segments are not stored in 
ascending order. Instead, in Mode C3*the left halves of the 
lines corresponding to two segments SI and S2 are stored as 
the first and second fields in frame Fl, and the right halves 
of the lines corresponding to two segments S5 and S6 are 20 
stored as the first and second fields in frame F2. The 
remaining segments S3, S4, S7, S8 are similarly stored as 
frames F3 and F4. 

In Mode C4, the left halves of the lines corresponding to 
two segments SI and S2 are stored as the first and second ^ 
fields in frame Fl, and the left halves of the remaining lines 
corresponding to segments S3 and S4 are stored as the first 
and second fields in frame F2. The right halves of the lines 
corresponding to segments S5 and S6 are stored as firame F3, 
and the right halves of the lines corresponding to segments 30 
S7 and S8 are stored as frame F4. 

Modes C3 and C4 have the advantages of allowing the 
direct display of a quadrant (two adjacent segments) of the 
image on a conventional sequential scan (non-interlaced) 
monitor, and in the event of discrepancies in or uncertainty 
about the information in the overlapping regions between 
quadrants, the adjacent quadrants may be viewed separately 
for comparison. Mode C3 presents the disadvantage of 
requiring a one frame delay in order to seam the quadrants 
properly, and Mode C4 requires a two frame delay. Mode C3 ^ 
is dierefore more expensive than Mode C2, and Mode C4 
more expensive than Mode C3. Mode C4 may provide an 
advantage in simultaneously viewing two vertically adjacent 
quadrants which represent a strip of an electronic image or 
tangible document which has a extremdy long length 
equivalent to a continuous roll of paper. 

In the second altemative (Mode C2) is to place each half 
of the stretched line sequentially one after another. The first 
vertical block or field would therefore have 256 lines of 
active video comprising 128 first halves alternating with 128 
second halves. This continues for 8 blocks or fields, at which 
point 4 frames have been stored. To retrieve an image stored 
in this mode, the added sync signals are removed and the 
second halves are delayed and joined to form continuous 
lines having a 12 Mhz bandwidth. 

When an image is retrieved in Mode B, the blocks or 
isegments corresponding to adjacent lines will seam together 
naturally because the separation between blocks is equiva- 
lent to a normal horizontal or vertical synchronization pulse. 60 
In the case of Une segments, the top quadrants will seam 
together with their adjacent bottom quadrants, however a 
more complex seaming technique is preferably utilized 
between horizontally adjacent quadrants. 

One technique for seaming horizontally adjacent quad- 65 
rants is overscanning, in which the first halves each line 
segment extend 2% beyond the midpoint of the individual 
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line segment to include a leading portion of the second half 
of the same line (in another segment and possibly in another 
frame), and the second halves begin 2% before the midpoint 
to include a trailing portion of the first half of the same line 
(also in another segment and possibly in another frame). The 
overlapping or redundant information may be compared in 
dc or ac content, time, and amplitude to match the seam, 
with servo loops being employed to provide the necessary 
degree of precision required in the manner discussed below. 
Higher frequency content and increasing grayscale depth 
will dictate the type of servo loops used. 

"Mode N" Operation 

Mode N refers generally to any operating mode which 
employs one or more capture devices 32 to capture, dissect, 
and seam together or reconstitute an electronic image cor- 
responding to a source document larger than a basic US 
letter-size page at "normal" resolution, or to a source docu- 
ment of any size that is captured and stored as more than four 
frames and more than eight segmwits. It may be readily 
appreciated from this discussion that it various embodiments 
may be designed around the use of a single capture device 
32 with the source document being moved or advanced to 
present different regions that would correspond to different 
segments of a larger electronic image or document, or that 
several capture devices 32 could be utilized as shown in 
FIGS. 10 or 12, either with or without movement or advanc- 
ing the source document. A line-array type capture device 32 
as shown in FIG. 13 may be more practical for some 
continuous feed applications in which the documents have a 
fixed width but variable length. 

In some applications, such as the example provided above 
relating to negotiable instruments or checks, an array of two 
capmre devices 32 such as shown in FIG. 11 are utilized to 
capture an images relating to the front and back faces of the 
source document, and those two images are subsequently 
placed together as a single half-page segment or frame. It 
may also be appreciated that several segments from several 
different capture devices 32 can be seamed together without 
regard to whether each of the segments represents a portion 
of a larger source document, thus allowing new documents 
to be created by assembling segments of other images 
together as though reconstituting and original electronic 
image. 

Seaming and Image Control Signals 

The raster synchronization discussed above that is added 
to the analog signal in order to define frames corresponding 
to the selected storage medium is a conventional fonnat that 
may be recognized by commercial and consumer video 
equipment, however such a format is only a **coarse" timing 
reference compared to the degree of accuracy or precision 
necessary to dissect and reconstitute electronic image docu- 
ments and still maintain the requisite level of qualitative 
integrity for the informational content described. 

The vertical and horizontal synchronization pulses 
embedded in standard RS-170 formatted signal (used with 
standard television display and recording) are unsuitable for 
seaming adjacent segments of an electronic image. Four 
types of image control signals are therefore utilized with the 
embodiments of the system 10 disclosed herein. 

The first two sets of signals are the horizontal and vertical 
raster synchronization pulses (and corresponding blanking 
intervals) corresponding to the camera output and frame 
formats discussed above. Referring to FIG. 14a, a 50 micro- 
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second sampling of a 12 Mhz information signal 118 is 
shown bracketed by a pair of 13.5 microsecond blanking 
intervals 120 and a pair of 5 microsecond negative-going 
synchronization pulses 122. Hie line thus has a duration of 
approximately 63.5 microseconds. When digitized or quan- 5 
titized at a 31 Mhz conversion clock rate this produces a 50 
microsecond digital signal as shown in FIG. 14b, which may 
be segmented and stretched into two halves which are each 
51.5 microseconds in length by reading from the memory 
buffer 60 at 15.5 Mhz, thus producing two spaced-apart lO 
signals as shown in FIG. 14c and an intervening dead zone 
124. The two spaced-apart signals are converted to analog 
signals as shown in FIG. l^d, with a portion constituting 
approximately 2% of each half line situated directly adjacent 
to the dead zone 124 accounting for the overlap 126 created 15 
by and utilized for overscanning. 

The third type of image control signal are calibration 
pulses 128 embedded as a part of the video signal just before 
and after the active video portions of the composite signal. 
The calibration pulses are positive-going pulses inserted 20 
immediately before and after the synchronization pulses for 
each line (or line segment). Each c^bration pulse preferably 
has a width (duration) on the order of 750 nanoseconds and 
a height (amplitude) of 0.35 volts DC when a 0.714 volt 
active video signal is utilized. Hie calibration pulses enstire ^5 
matching of the DC levels and amplitudes of adjacent 
segments, incremental gain calibradon between adjacent 
fines (or Ime segments) of the image and against a set level 
for each line throughout the subject image and prior or 
subsequent images, and also provide an intermediate timing 30 
reference that has a resolution greater than the conventional 
synchronization signals but less that the pilot signal. 

Gain calibration is accomplished by subtracting black 
from gray, with true black being clamped at zero. If a 
calibration pulse is read at 0.4 V, for example, a servo 
correction brings that line down to 0.35 V to provide an 
accurate gray level for each "pixel" within the line. The 
calibration pulse at the end of any line may be directly 
compared with the calibration pulse at the beginning of any 
adjacent line along the corresponding seam. Any divergence 
from the normal calibration amplimde results in shifting the 
level of the corresponding line so that each line exactly 
matches the adjacent lines and is set relative to a fixed base 
level, and the servo loops used for the calibration adjust- 
ments are therefore completely independent of informa- 
tional content in the image. 

It may be appreciated that the calibration pulses 128 
provide reference to a predetermined "absolute gray" that 
also corresponds to a similar value for monitors and other 
devices, with the focus of the embodiments discussed being 
on a predetermined area of interest or range of grayscale 
levels that are conmionly encountered in tangible source 
documents of the type used for business records, financial 
transactions, and so forth. The amplitude of the calibration 
pulses 128 may be set at any desured value to provide control 
over the location of the examining area within the entire 
grayscale range by shifting the median grayscale of the 
image upward or downwardly compared to an "absolute 
gray" value expected by the monitor or other device, while 
still permitting "on the fly" correction around the calibration 
pulses 128 for line-to-lme nd segment-to-segment precisioa 

A conventional negative-going synchronization pulse 130 
is then inserted between the calibration pulses 128, as shown 
in FIG. 14/ 65 

The fourth type of image control signal is a low level 
(CW) pilot signal 132 or carrier tone that extends continu- 
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ously throughout the stored analog signal, is synchronous 
with the calibration pulses 128, and is harmonically related 
to the clock speed, ThQ pilot signal 132 must be high enough 
frequency to permit harmonization with all other synchro- 
nizing or timing signals and pulses, and therefore compat- 
ible with all divisors of the main system clock rate, and is 
preferably at or near one half of the write clock frequency. 
The pilot signal 132 should also be a firequency just above 
the predetermined bandwidth of the storage medium, but 
sufGciently distinguishable that a bandpass filter can be used 
to strip the pilot signal 132 without clipping active video 
content 118 from the signal. Since the operational spectrum 
for conventional laser disc medium faUs off rapidly above 
the 6 Mhz bandwidth, a low level pilot signal 132 on the 
order of 6.89 Mhz has proven suitable for use with conven- 
tional LDRs, and may be filtered out prior to display of the 
electronic image to prevent visual disruption of the dis- 
played image. The pilot signal 132 therefore provides many 
times more vernier than the standard synchronization pulses 
for time-based corrections in positioning and aligning adja- 
cent segments of the image when seaming those segments 
together along horizontal seam lines (parallel witii raster 
lines) or vertical seam lines (perpendicular to raster lines). 
The pilot signal 132 is also necessary to provide a time- 
based correction capability to corxqicnsate for timing errors 
in conventional LDRs, and the use of the recovered pilot 
signal 128 which is retrieved as a part of the stored signal to 
generate the write clock as data is written to the memory 
buffer 60 allows the use of this modality in connection with 
other storage and communications systems that produce 
time-based enrors or which do not regulate absolute timing 
relative to an extemal reference or source. 

When the pilot signal 132, calibration pulses 128, and 
synchronization pulses 130 of FIGS. 14c-14^ are summed 
with the active signal portion of FIG. 14£i a composite 
analog waveform 134 results. As shown in FIG. 14^ that 
waveform generally follows and oscillates about the 
summed control signals 128, 130, 132. 

To reconstitute the image, the composite signal 134 of 
FIG. 14h is retrieved from the storage medium, and a square 
wave from the write clock at 15.5 Mhz (derived from a 
recovered 6.89 Mhz pilot signal) is used to quantitize or 
digitize the signal and write the signal to the memory buffer 
60 in the same form as shown in FIG. 14c. The 'left hand" 
segment of the digitized signal is delayed to overiap with the 
"right hand" segment, and the two segments are seamed 
together as shown in FIG. 14k and the digital array is 
converted to an analog signal with raster synchronization 
and blanking intervals added as shown in FIG. 14a, thus 
permitting the storage, transmission, or display of the elec- 
tronic image. In the four versions of Mode C operation, the 
delay will be a line, field, or frame delay depending upon the 
mode. 

For an 8-bit grayscale image, the processed signal does 
not correspond to a 1:1 quantization that would produce a 
true 256 level grayscale range for the electroruc image. 
Instead, it has proven suitable in the embodiments discussed 
to incorporate an approximately 20% margin to provide 
overhead for calibration adjustments and offset. As such, the 
effective grayscale range would be on the order of 200 to 
210 levels, which accommodates the vast majority of inter- 
active document processing applications including high 
resolution medical imaging for X-rays and MRIs, and is 
state-of-the-art for commercially available LDRs. Applica- 
tions requiting enhanced grayscale differentiation may result 
in sacrificing a portion of the margin, or increasing the bit 
depth of the system. 
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Transmission Pathways 

The transmission of an electronic image as an analog 
signal provides significant improvement in transmission 
rates compared with even the fastest conventional digital ^ 
transmission by modem or facsimile or over network lines. 
However, it may be appreciated that the segmenting of the 
electronic image in the memory buffer 60 and conversion to 
a resultant analog signal of appropriate bandwidth permits 
additional increases in transmission rates over conventional 
transmission pathways having limited bandwidths, as well lO 
as high speed transmission pathways having larger band- 
widths. 

It is important to note when considering conversion of an 
analog signal for transmission that the parameters for trans- 
mission over a digital transmission pathway are traditionally 15 
specified in Mhz clock speeds. According to the Nyquist 
theorem, a perfect system free of filtering imperfections 
would permit the transmission of an analog signal having a 
frequency (R) referred to as the Nyquist frequency at no less 
than twice (2R) the clock rate which is referred to as the 20 
sampling or Nyquist rate. In the preferred embodiments 
discussed above, the Nyquist frequency of the analog signal 
from the capture device 32 and from the initial digital-to- 
analog conversion is on the order of 12 Mhz, whereas the 
sampling rate uses a 3 1 Mhz clock rate. Consequently, this ^ 
produces a Nyquist ratio of approximately 2.6 between the 
clock rate and analog Nyquist frequency. This conversion 
ratio also provides an estimate for the effective analog 
bandwidth that may be allotted to a specific digital pathway. 
As such, if one were transmitting a 12 Mhz analog signal 
over a 4 Mhz clock rate transmission pathway in a perfect 
system (where the Nyquist ratio is 2 and the maximum 
bandwidth is 2), it would be necessary to either slow the 
analog signal down by a factor of 6 times and transmit 
continuously, or divide the signal into six components and 
transmit in parallel. 

Utilizing the system 10 of this invention, it is possible to 
accomplish either of these processes in a way that is optimal 
for the particular application and transmission pathway. The. 
actual bandwidtii of the analog signal may be reduced to a 40 
lower bandwidtii tiiat corresponds to a digital clock rate at or 
below the maximum transmission clock rate of the digital 
transmission pathway, and then transmitted at that speed 
(which would be slower than if transmitted at the original 
bandwidtii over a line having the same effective bandwidtii 45 
or clock speed.) Alternately, tiie electronic image may be 
divided into a plurality of segments each having an effective 
bandwidth correspondiDg to the maximum transmission 
clock rate of a particular digital pathway, and then trans- 
mitted in parallel and reconstituted. In a transmission path- 50 
way having more than one parallel lines or channels (such as 
a conventional ISDN transmission pathway comprising 
three parallel channels of which two are allocated for digital 
transmission), tiie segments of tiie electronic image in tiae 
memory buffer 60 can be read in parallel and transmitted 55 
simultaneously on the separate digital channels or fines. 
Again, the bandwidth of tiie analog signal for each channel 
is optimized according to the operating parameters of that 
channel. Subsequent frequency conversion of the type 
obtained using a line multiplexer may also be applied in go 
applications such as fiber-optic transmission, permitting a 
multipHcity of segments of an electrotuc image to be trans- 
mitted simultaneously. The analog signal may also be 
divided accordingly and transmitted over an analog trans- 
mission pathway at the maximum bandwidth permitted. ^5 

In tiieir simplest embodiments,.the high speed network 28 
and transmission pathway 26' are broadband cables capable 
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of transmitting television-type signals, utilizing a conven- 
tional handshake recognition and latch to lock out non- 
requesting nodes finm the communication procedure once a 
request is received by the mainframe or controller and an 
instruction to retrieve and transmit an image is received and 
completed by the electronic image server 30 and mass 
storage device 34. Image requests, sequencing or cataloging 
information, and ready-state or other control signals may be 
transmitted over a conventional digital network such as a 
Novell or Ethernet system. 

Because the transmission bandwidths and number of 
available chaimels or pathways are fixed for conventional 
communications networks, tiie system 10 may be optimized 
to permit transmission on these existing communication 
lines, including satellite and microwave transmission, mul- 
tichannel RF television, as well as transmission over home- 
broadcast cable television systems. The transmission opera- 
tion is also completely compatible with and transparent to 
any conventional communications-related security technol- 
ogy, such as those employing moditiated line scrambling 
devices and signal encryption algorithms. 

It may be readily appreciated that the various embodi- 
ments and modes of operation discussed above constitute 
only representative examples of the optimization of the 
disclosed method to: (1) a specific size and type of source 
document; (2) a particular operating environment; (3) a 
predetermined level of informational content or resolution 
for the electronic image; and (4) tiie utihzation of selected 
devices and (equipment for initial image capture, CPU bus 
and interface, storage medium, and transmission pathways 
based upon existing commercial availability and cost. 

It is understood that the systems for practicing the dis- 
closed method may be optimized according to many differ- 
ent parameters beyond informational content and processing 
time, however it is expected that tiiese two parameters will 
remain most significant for practical commercial applica- 
tions. It is further understood Uiat many factors will affect 
the design of alternate modes of operation or further 
embodiments of the system, such as: the selection of alter- 
nate devices or equipment; the continuing refinement and 
introduction of new capture, storage, and transmdssion tech- 
nologies; the operational guidelines imposed for accom- 
plishing specified tasks for certain applications; tiie nature 
and limitations of the operating environment; the modifica- 
tions or adaptations involved in applying this method to 
electronic content; the transition between different televi- 
sion, video, and communications standards in this and other 
countries; as well as the peculiar emphasis that individuals 
or users may be placed on etherial considerations such as 
cost, complexity, security, or other aesthetic values affecting 
basic design feamres. 

While the preferred embodiments of the above electronic 
document transmission, storage, and retrieval system 10 has 
been described in detail with ref^ence to the attached 
drawing Figures, it is understood that various changes and 
adaptations may be made in the electronic document trans- 
mission, storage, and retrieval system 10 without departing 
from the spirit and scope of tiie appended claims. 

What is claimed is: 

1. A metiiod for tiie storage and retrieval of a two- 
dimensional electronic image expressible as a digital anay 
or bitmap for tiie purpose of interactive document process- 
ing, said method being utilized in association with a storage 
device having a storage medium, said method comprising 
the steps of: 

writing at least a portion of tiie digital array or bitmap 
corresponding to the two-dimensional electronic image 
to an output memory; 
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leading said digital content from said output memory; 
converting said digital content to a first analog signal as 

said digital content is read from said output memory; 
adding a plurality of image control signals to said first 

analog signal to define a format, said format defining a ^ 

frame; 

selectively transmitting said first analog signal over the 
transmission pathway to the remote location or storing 
said fint analog signal as at least one said frame on the 
storage device, the storage device being capable of 
receiving and storing said first analog signal on the 
storage medium along with a multiplicity of like 
frames, the storage device further being capable of 
randomly accessing and selectively retrieving and out- 
putting a separate analog signal corresponding to said 
first analog signal stored as at least one said frame from 
among said multiplicity of lie frames, said separate 
analog signal including said plurality of image control 
signals; 

retrieving said separate analog signal including said plu- 
rality of image control signals from the storage device; 

stripping said plurality of image control signals from said 
separate analog signal retrieved from the storage 
device; 

converting said separate analog signal retrieved from the 
storage device to a digital signal; 

writing said digital signal to an input memory such that 
the digital array or bitmap corresponding to the two- 
dimensional electronic image is reconstituted in said 
input memory, 

whereby the electronic image may subsequently be uti- 
lized in one or more document processing operations 
including the transformation, presentation, representa- 
tion, transmission, or storage and retrieval of the elec- 
tronic image, 

wherein the plurality of image control signals includes 
a plurality of raster synchronization pulses and cor- 
responding blanking intervals, at least one pilot 
signal, and at least one set of calibration pulses. 

2. The method of claim 1 wherein the storage medium has 40 
a predetermined bandwidth and wherein the frequency of the 
pilot signal is at or slightiy above said predetermined 
bandwidth of the storage mediunL 

3. Hie method of claim 2 wherein the predetermined 
bandwidth of the storage medium is on the order of 6 Mhz 
and the, pilot signal is on the order of 7 Mhz. 

4. The method of claim 1 wherein the pilot signal is a 
continuous signal which extends throughout the separate 
analog signal. 

5. The method of claim 1 wherein the format defining the 
frame is a raster composed of N scarming lines at a prede- 
termined scan rate, each of said scanning lines having an 
active portion corresponding to an informational content of 
the two-dimensional electronic image, and further wherein 
the at least one set of calibration pulses include a first 
calibration pulse inserted before the active portion of each of 
the scanning lines and a second calibration pulse inserted 
after the active portion of each of the scanning lines. 

6. Hie method of .claim 5 wherein the first calibration 
pulse and the second calibration pulse each have a duration 
and an amplitude, such that said amplitude of the first 
calibration pulse of a first scarming line may be compared 
with either said amplitude of the first calibration pulse or 
said amplitude of the second calibration pulse of a like 
scarming line or with a fixed amplitude. 

7. The method of claim 6 wherein each of the scarming 
lines has a gain associated therewith, and wherein said gain 
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of a selected scarming line may be adjusted in response to a 
comparison between the first calibration pulse of said 
selected scanning tine and the second calibration pulse of a 
like scanning line or with a fixed amplitude. 

8. The method of claim 6 wherein the duration of the first 
calibration pulse and the second calibration piilse is on the 
order of 750 nanoseconds. 

9. The method of claim 6 wherein the amplimde of the 
first calibration pulse and the second calibration pulse is on 
the order of 0.35 volts. 

10. The method of claim 1 wherein the at least one set of 
calibration pulses provides an intermediate timing reference 
that has a resolution greater than the plurality of raster 
synchronization pulses and corresponding blanking intervals 
and less than the pilot signal. 

11. The method of claim 1 wherein the digital array or 
bitmap corresponding to the two-dimensional electronic 
image is composed of X lines, and wherein the format 
defining the frame is a raster composed of N scanning lines 
at a predetermined scan rate. 

12. The method of claim 11 wherein the finame is com- 
posed of X/2 lines, and a time interval dictated by a quantity 
of 2N-X excess scanning lines and the predetermined scan 
rate is utilized for containing the plurality of image control 
signals. 

13. The method of claim 12 wherein X is on the order of 
1024, N is on the order of 525, and the predetermined scan 
rate is on the order of 60 hz. 

14. The method of claim 1 wherein the method further 
comprises the step of: 

sectioning the digital array or bitmap corresponding to the 
two-dimensional electronic image into a plurality of 
segments prior to or during the step of writing the 
digital content to the output memory. 

15. The method of claim 14 wherein the number of the 
frames is one, and the number of the plurality of segments 
is two. 

16. The method of claim 14 wherein the number of the 
frames is two, and the immber of the plurality of segments 
is four. 

17. The method of claim 16 wherein the digital array or 
bitmap corresponding to the two-dimensional electronic 
image is composed of X lines, and wherein each segment 
corresponds to a block of X/4 adjacent lines. 

18. The method of claim 17 wherein the plurality of 
segments includes a first segment, a second segment, a third 
segment, and a fourth segment, said first segment being 
adjacent to said second segment and said third segment 
being adjacent to said fourth segment, and wherein the 
number of the frames includes a first frame and a second 
frame, said first segment and said second segment being 
stored in said first frame and said third segment and said 
fourth segment being stored in said second frame. 

19. The method of claim 14 wherein the portion of the 
digital array or bitmap is written to tiie output memory at a 
write speed, and the digital content from the output memory 
is read at a read speed, said read speed being generally equal 
to one half said write speed. 

20. The method of claim 19 wherein the number of the 
frames is four, and the number of the plurality of segments 
is eight. 

21. The method of claim 20 wherein the digital array or 
bitmap corresponding to the two-dimensional electronic 
image is composed of X lines, and wherein each one of the 
plurality of segments corresponds to X/8 lines. 

22. The method of claim 20 wherein the digital array or 
bitmap corresponding to the two-dimensional electronic 
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image is composed of X lines, each of said lines having a 
first half and a second half, and wherein each one of the 
plurality of segments corresponds to XyS of said first halves 
of said lines or X/8 of said second halves of said lines. 

23. The method of claim 22 virherein the plurality of 5 
segments includes a first segment, a second segment adja- 
cent to said first segment, a third segment adjacent to said 
first segment, a fourth segment adjacent to said third seg- 
ment and said second segment, a fifth segment adjacent to 
said third segment, a sixth segment adjacent to said fifth 
segment and said fourth segment, a seventh segment adja- 
cent to said fifth segment, and an eighth segment adjacent to 
said seventh segment and said sixth segment, each of said 
first segment, said third segment, said fifth segment, and said 
seventh segment being composed of X/8 of the first halves, 
each of said second segment, said fourth segment, said sixth 
segment, and said eighth segment being composed of X/8 of 
the second halves, wherein the number of the frames 
includes a first frame, a second frame, a third frame, and a 
fourth frame, said first segment and said second segment 
being stored in said first frame, said third segment and said 20 
fourth segment being stored in said second frame, said fifth 
segment and said sixth segment being stored in said third 
frame, and said seventh segment and said eighth segment 
being stored in said fourth frame. 

24. The method of claim 22 wherein the plurality of 25 
segments includes a first segment, a second segment adja- 
cent to said first segment, a third segment adjacent to said 
second segment, a fourth segment adjacent to said third 
segment, a fifth segment adjacent to said first segment, a 
sixth segment adjacent to said fifth segment and said second 30 
segment, a seventh segment adjacent to said sixth segment 
and said third segment, and an eighth segment adjacent to 
said seventh segment and said fourth segment, each of said 
fint segment, said second segment, said third segment, and 
said fourth segment being composed of X/8 of the first 35 
halves, each of said fifth segment, said sixth segment, said 
seventh segment, and said eighth segment being composed 

of X/8 of the second halves, wherein the number of the 
frames includes a first frame, a second frame, a third frame, 
and a fourth frame, said first segment and said second 40 
segment being stored in said first frame, said third segment 
and said fourth segment being stored in said second frame, 
said fifth segment and said sixth segment being stored in said 
third frame, and said seventh segment and said eighth 
segment being stored in said fourth frame. 45 

25. The method of claim 24 wherein the first frame and 
the second frame are located in sequence and adjoining one 
another on the storage medium. 

26. The method of claim 24 wherein the first frame and 
the third firame are located in sequence and adjoining one 50 
another on the storage medium. 

27. The method of claim 14 wherein the method fiirther 
comprises the step of: seaming the plurality of segments 
together prior to or during the step of writing the corre- 
sponding digital content to the input memory such that the 55 
digital army or bitmap corresponding to the two-dimen- 
sional electronic image is reconstituted in the input memory. 

28. The method of claim 27 wherein the digital array or 
bitmap corresponding to the two-dimensional electronic 
image is composed of a plurality of lines, an individual line go 
of said plurality of lines including a first half and a second 
half, and wherein seaming the plurality of segments together 
comprises: 

reading either the first half of said individual line plus a 
leading portion of said second half of said individual 65 
line or said second half of said individual line plus a 
trailing portion of said first half of said individual line; 
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comparing said leading portion of said second half of said 
individual line with said trailing portion of said first 
half of said individual line to determine an offset 
between said first half and said second half of said 
individual line; and 

adjusting said first half or said second half to eliminate 
said offset. 

29. The method of claim 28 wherein the leading portion 
and the trailing portion of the individual line arc disposed in 
two different ones of the plurality of segments and m one of 
the frames. 

30. The method of claim 28 wherein the leading portion 
and the trailing portion of the individual line are disposed in 
two different ones of the plurality of segments and two 
different ones of the frames, 

31. A method for the storage and retrieval of a two- 
dimensional electronic image expressible as a digital array 
or bitmap for the purpose of interactive document process- 
ing, said method being utilized in association with a storage 
device having a storage medium, said method comprising 
the steps of: 

writing at least a portion of the digital array or bitmap 

corresponding to the two-dimensional electronic image 

to an output memory; 
reading said digital content from said output memory; 
converting said digital content to a first analog signal as 

said digital content is read from said output memory; 
adding a plurality of image control signals to said first 

analog signal to define a format, said format defining a 

frame; 

selectively transmitting said first analog signal over the 
transmission pathway to the remote location or storing 
said first analog signal as at least one said frame on the 
storage device, the storage device being capable of 
receiving and storing said first analog signal on the 
storage medium along with a multiplicity of Uke 
frames, the storage device further being capable of 
randomly accessing and selectively retrieving and out- 
putting a separate analog signal corresponding to said 
first analog signal stored as at least one said frame from 
among said multiplicity of lie frames, said separate 
analog signal including said plurality^ of image control 
signals; 

retrieving said separate analog signal including said plu- 
rality of image control signals from the storage device; 

stripping said plurality of image control signals from said 
separate analog signal retrieved from the storage 
device; 

converting said separate analog signal retrieved from the 
storage device to a digital signal; 

writing said digital signal to an input memory such that 
the digital array or bitmap corresponding to the two- 
dimensional electronic image is reconstituted in said 
input memory, 

whereby the electronic image may subsequently be uti- 
lized in one or more document processing operations 
including the transformation, presentation, representa- 
tion, transmission, or storage and retrieval of the elec- 
tronic image, 

wherein the two-dimensional electronic image is captured 
from a tangible document and converted to the digital anray 
or bitmap, said method ftirther comprising the steps of: 
capturing an initial image in a capture device, said initial 
image corresponding to a camera raster composed of X 
scanning lines having a predetermined scan rate, said 
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capture device producing an analog output signal cor- 
responding to sequentially outputting said X scanning 
lines at one half said predetermined scan rate, said 
analog output signal including a camera raster synchro- 
nization; 

stripping said camera raster synchromzation &om said 

analog output signal; 
converting said analog output signal to a digital input 

signal; and 

writing said digital input signal to a capture memoiy to 
form the digital array or bitmap corresponding to the 
two-dimensional electronic image, said digital array 
being composed of 2X lines. 

32. The method of claim 31 wherein the analog output 
signal from the capture device has an initial bandwidth, and 
wherein the method further comprises the steps of: 

converting the initial bandwidth of the analog output 
signal to a resultant bandwidth prior to storing or 
transmitting the first analog signal; and 

converting said resultant bandwidth to the initial band- 
width subsequent to retrieving the separate analog 
signal or receiving the first analog signal. 

33. The method of claim 32 wherein the initial bandwidth 
is on the order of 12 Mhz or greater and the resultant 
bandwidth is on the order of 6 Mhz. 

34. The method of claim 31 wherein the capture device is 
a camera having a normal operating scan rate for producing 
the camera raster composed of the X scanning lines having 
the predetennined scan rate, and wherein the method further 
comprises the step of: 

driving the camera at an actual scan rate generally equal 
to one haft the normal operating scan rate. 

35. The method of claim 31 wherein the method further 
comprises the step of: 

multiplexing the digital input signal prior to or during the 
step of writing the digital input signal to the capture 
memory. 

36. The method of claim 31 wherein the capture memoiy 
and the input memory are defined by one semiconductor 
memory. 

37. The method of claim 31 wherein the initial image is 
composed of 1024 pixels in a vertical direction by 1000 
pixels in a horizontal direction, and wherein the capture 
memory is a 1024 by 2048 bit memory. 

38. The method of claim 31 wherein the capture memory 
is composed of two banks of eight 128 kbit memory. 

39. An apparatus for the storage and retrieval of a two- 
dimensional electronic image existing as a digital array or 
bitmap in a memory for the purpose of interactive document 
processing by a user, said .apparatus comprising: 

a computer, said computer including a communications 
pathway, an electronic image processing interface 
operatively connected to said conrniunications path- 
way, and the memory, the memoiy being operatively 
connected to said conrniunications pathway; 

a storage device, said storage device being operatively 
connected to said, electronic image processing interface 
and including a storage medium capable of receiving 
and storing an analog signal in a designated form in a 
predetermined format defining a ftame; and 

said electronic image processing interface being capable 
of either 

selectively storing the two-dimensional electronic 
image to the storage device by converting the digital 
array or bitmap in the memory to a first analog signal 
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as said digital content is being read from the 
memory, and adding a plurality of image control 
signals to said first analog signal in the predeter- 
mined format corresponding to at least one of the 
frames, or 

selectively retrieving the two-dimensional electronic 
image from the storage device by retrieving the at 
least one of die frames as a separate analog signal 
including said plurality of image control signals from 
said storage device, stripping said image control 
signals from said separate analog signal, converting 
said separate analog signal to a digital signal, and 
writing said digital signal to the memory such that 
the digital array or bitmap corresponding to the 
two-dimensional electronic image is reconstituted in 
the memory, 
said apparatus further comprising: 
a transmission network said transmission network 
being operatively connected to Uie electronic image 
processing interface, wherein the transmission net- 
work is capable of transmitting the first analog signal 
including the plurality of image control signals to a 
remote location whereby the first analog signal may 
be received by a second computer having an second 
electronic image processing interface and a second 
memoiy, said second electronic image processing 
interface being at least capable of stripping said 
plurality of image control signals from said first 
analog signal, converting said first analog signal to a 
digital signal, writing said digital signal to said 
second ndemory associated with said second elec- 
tronic image processing interface such that the digi- 
tal array or bitmap corresponding to the two-dimen- 
sional electronic image is reconstituted in the second 
memory of the second computer. 

40. The apparatus of claim 39 wherein the electronic 
image processing interface is capable of both selectively 
storing the two dimensional electronic image to the storage 
device and selectively retrieving the two-dimensional elec- 
tronic image from the storage device. 

41. The apparatus of daim 39 wherein the storage device 
is operatively connected to the electronic image processing 
interface through the commimications pathway, 

42. The apparams of claim 39 wherein the electronic 
image processing interface is further capable of sectioning 
the digital array or bitmap conesponding to the two-dimen- 
sional electronic image in the memory into a plurality of 



43. The apparatus of claim 42 wherein the transmission 
network includes a plurality of transmission pathways and 
each one of the plurality of segments may be transmitted in 
parallel over said plurality of transmission pathways. 

44. The apparams of claim 42 wherein the transmission 
network includes at least one transmission pathway having 
a transmission bandwidth, and wherein the number of the 
plurality of segments is determined such that the first analog 
signal corresponding to each of said plurality of segments is 
within said transmission bandwidth. 

45. The apparatus of claim 39 wherein the first analog 
signal has an initial bandwidth, and wherein the electronic 
image processing interface is further capable of converting 
said initial bandwidth to a resultant bandwidth. 

46. The apparatus of claim 45 wherein the transmission 
network includes at least one transmission pathway having 
a transmission bandwidth, and wherein the resultant band- 
width is withm said transmission bandwidth. 

47. The apparatus of claim 39 wherein the transmission 
network is operatively connected to the electronic image 
processing interface through the communications pathway. 
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48. The apparatus of claim 39 wherein the designated 
form in which the analog signal is stored is on a digital 
medium such as an optical laser disc. 

49. A method for tiie storage and retrieval of a two- 
dimensional electronic image expressible as a digital array 5 
or bitmap for the purpose of interactive document process- 
ing, said method being utilized in association with a storage 
device having a storage medium, said method comprising 
the steps of: 

writing at least a portion of the digital array or bitmap 10 

corresponding to the two-dimensional electronic image 

to an output memory; 
reading said digital content from said output memory; 
converting said digital content to a first analog signal as 

said digital content is read from said output memory; 
adding a plurality of image control signals to said first 

analog .signal to define a format, said format defining 

a frame; 

selectively transmitting said first analog signal over the 20 
transmission pathway to the remote location or storing 
said first analog signal as at least one said frame on the 
storage device, the storage device being enable of 
receiving and storing said first analog signal on the 
storage medium along with a multiplicity of like 25 
frames, the storage device further being capable of 
randomly accessing and selectively retrieving and out- 
putting a separate analog signal corresponding to said 
first analog signal stored as at least one said frame from 
among said multiplicity of lie flames, said separate 30 
analog signal including said plurality of image control 
signals; 



retrieving said separate analog signal including said plu- 
rality of image control signals from the storage device; 

stripping said plurality of image control signals from said 
separate analog signal retrieved from the storage 
device; 

converting said separate analog signal retrieved from the 
storage device to a digital signal; 

writing said digital signal to an input memory such that 
the digital array or bitmap corresponding to the two- 
dimensional electronic image is reconstituted in said 
input memory, 

whereby the electronic image may subsequently be uti- 
lized in one or more document processing operations 
including the transformation, presenution, representa- 
tion, transmission, or storage and retrieval of the elec- 
tronic image, 

wherein the first analog signal has an initial bandwidth, and 
wherein the method further comprises the steps of: 
converting the initxal bandwidth of the first analog signal 
to a resultant bandwidth prior to storing or transmitting 
the first analog signal; and 
converting said resultant bandwidth to the initial band- 
width subsequent, to retrieving the separate analog 
signal or receiving the first analog signal. 

50. The method of claim 49 wherein the initial bandwidth 
is on the order of 12 Mhz or greater and the resultant 
bandwidth is on the order of 6 Mhz or less. 

51. The method of claim 1 wherein both the output 
memory and the input memory are defined by one semicon- 
ductor memory. 
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1.3.1. 1 Les diffirentes categories d'iditeurs interactifs 




Lei Miteurs de texte interaaifs peuvent itre classes en quatre 
categories: ies editeurs orient^s par la syntaxe, les editeun 
orient^s par ia struaure, les editeurs de texte ou processeurs de 
texte, etles edlteurs/formatteurs i c omoosan ts graphiques. 



Les premiers correspondant aux outils qui se diveioppent le plus 
so uvent autourde la programmation. Ilssonten general lies i 
un langagedeprogrammationetAsescaracteristiques. lis V^^f 
permetter^t au progammeur d' accroftre sa productivity en 
I'aidant a presenter ses programmes correctement eten operant 
des contrAles syntaxiques qui r eduisent ie temps de rnise au 
point. Au deli de leun fonctionnalitfes de traitement de texte. 
its integrent en fait certaines fonctions d6volue$ habituellemnt 
au c ompilateur ou i I'interprdteur. 11 



Les seconds c herchent i exploiter la structure naturelle des 
documents. AInsi une lettre est en sous parties mod6lt$able sous 
la forme d'un arbre contenant: rephtfite, la date, le nom et 
Tadresse du d estinataire . le cores de la lettre. des parag raphes, 
des sous-paragraphes, uneformulede politesse, une signature. 







FIG. 3 
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1.3.1,1 Les diffirentes categories d'iditeurs tnteractifs 

Les editeurs de texte interactifs peuvent *tre classes en quatre 
categories: les Editeurs orient^s par la syntaxe, les Miteurs 
orient^s par la suucture, les 6diteun de texte ou proceueurs de 
terte, et les ^diteurs/formatteurs i c omoosan ts graphiques. 

Les premiers correspondant aux outii rqui se ddveioppent le plus 
souvent autour de la programmation. Its sent en general lifs i 
un langage de programmation et & set caracti ristiques. lis \ 
per tnettent au progammeur d' accroUr e sa productivity en ' ' 
Taidant ^ pr^enter ses programmes correctement et en operant 
des contr6les syntaxiques qui r ^dutsent le temps de mise au 
point. Au delA de ieun fonctionnalit^s de traitement de texte, 
ils int^grent en fait certaines fonctions d dvolues habituellemnt 
au comottateur ou h I'interpriteur. V a4 

Les seconds cherchent A exploiter 4a structure naturelle des 
documents. A Insi une lettre est en sous parties modilisabie sous 
la forme d'un arbre contenant: P en-tfite . la date, le nom et 
I'adresse du dgstmataire, le c orqs de la lettre, des paragraphes. 
des sous-paragraphes, une formuie de o olitesse . une signature. 
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METHOD AND APPARATUS FOR tal information related thereto is paxticularly difficult when 

SUPPLEMENTING SIGNIFICANT PORTIONS dealing directly with bit mapped document images rather 

OF A DOCUMENT SELECTED WITHOUT than with character code representations (such as ASCII for 

DOCUMENT IMAGE DECODING WITH text images). In die past, perfectly recognizable scanned text 

RETRIEVED INFORMATION 5 has been treated as being interchangeable with electronicaUy 

stored character code files, rather than as a special problem 

ThisisaContinuationofappUcationSer. No. 07/795,419 domain. However, in contrast to ASCII text files, which 

filed Nov. 19, 1991. now abandoned. permit users to perform operations such as Boolean alge- 

laraic Iccy word searches to locate text of interest, text 

BACKGROU>fD OF THE INVENTION lo information that is scanned without decoding is difficult to 

1. Cross-Refocnccs to Related AppUcations retrieve, without exhaustive viewing of each document, or 
The foUowingconcurrenUy filed and related U,S. Patent "^'^T summaries of the documents for 

Applications are hereby cross referenced and incorporated f purposes. Of course, document viewing or creation of 

by reference in their entir^. ^Method and Apparatus for ^^^^^ summanes requires extensive human effort 

Dctamining Boundaries of Wcards in Texr lo Huttenlocher *^ Examples of retrieval techniques that rely upon locating 

ct al., U.S. patent application Ser. No. 07/794392 now U.S. usefid terms in a document can be found in Salton and 

Pat No. 5321,770. McGill. Introduction To Modem Information Retrieval^ 

"Detecting Function Words Without Converting A Docu- McGraw-HiU, Inc.. 1983. Thus, techniques exist for com- 
ment to Character Codes" to Bloomberg et at. U.S. patent "^"^^ matches, locating the most frequent noun 
^plication Ser. No. 07/794,190 now abandoned. P^^^ ^ ^ composing stop-lists of words which arc 

"AMcthodofDerivingWordshapes for Subsequent Com. ^Iv^C.^ In^t'''' V kT^^ "^ i^f^n^aUon 

parison- to HuttcnlochcT et al.. uXpatent appHcation Ser. '^^'^^ "^^^^ tecfamques genendly 

No. 07/794391 now abandoned. "^^^^^'^^ <P^^y lecogmzable) text 

**Method and Apparatus for Determining the Frequency of 25 ^' References 

Words in a Document wifeout Document Image Decoding" '*-972349 to Klcinbcrger describes a cora- 

to Cass et al., U.S. patent application Ser. No. 07/796,173 puterized informaaon retrieval system and method formed 

now U S Pat No 5 208 969 ' ^ textbase of tfcxts of variable length and content. The 

*13ptical Word R^gnition By Examination of Word f^^ ^ ^^"^"^^ basis of Boolean 

Shape" to Huttenlocher et al., U.S. patent application Ser. K. key words assoc«ited with the texts. 

No. 07/796,119 now abandoned. When a group is retrieved from such a search, die system 

... . ^ J ei. ^ automatically segregates the texts based on the presence or 

'Mc^od for Comparing Word Shapes^ to Huttenlocher et ^ crit^^-key keyword selected to s^^gate the 

at, US. patent apphcaUon Ser. No. 07/795,169 now aban- sul^groups. THe same criteria key ^alysis can 

H then be applied recursively to the subgroups. The resulting 

A Method and Apparatus for Image Hand Markup subgroups are then displayed to the user in a hierarchical 

Dacction'*,U.S. patent application Ser. No. 07/794^75 now display to illustrate fte relationships among the texts. A 

US. Pat No, 5^1,011. string comparison routiDe is also described to search for 

2. Field of the Invention similar keywords. 

This invention relates to improvements in methods and ^ U.S. Pat. No. 4,985.863 to Fujisawa et al. describes a 

apparatus fcff electronic document processing, and more document storage and retrieval system and a method of 

paiticulariy to in^irovements in methods and ^)paratus for document retrieval that stores a portion of characters for 

automatically selecting semantically significant words, ou^utting and also stores the document in the form of an 

diaracters, images, or image s^ments in a document image image for retrieving. A retrieval request for a text is made 

without first decoding the document or othearwise under- 45 using a proper nuniber of the text or a special symboL The 

standing the information in the document, and augmenting document image can then be retrieved and stoi^ or decoded 

the document with additional retrieved information relating to character codes for ou^utting. Character recognition is 

to the selected words, characters, images, or image seg- performed to recognize a retrieval key of a document before 

retrieval, although the actual retrieval or transfer of the 

3. References and Background 50 document does not require coix4>lete (^aracter recognition. 

One objective of coaster based electronic document U.S. Pat. No. 5,010,581 to Kanno describes a data pro- 
processing is to facilitate the user's access to and under- cessing q>paratus for retrieving abstract and original image 
standing of the information contained in a document or abstract image is an abbreviated form of the 
ooipusofdocuments. However, in many cases, such as with original image. The q^atus includes input means for 
a document in a language or foim (e.g., non-Braille text for 35 inputting an original image; first memory means for tem- 
a blind user) which is foreign to the user, the user needs pmrily storing the original image ii^ut by the input means; 
additional information or translation of the document in drafting means for drafting an abstract image of the original 
order to obtain any understanding of the document. In other image; second memory means for storing the original and 
cases, even after a user reaches a level of understanding abstract images; and retrieval means for retrieving the 
about a document or group of documents, the user often 60 abstract image based on retrieval data corresponding to both 
desires to obtain supplemental information with which to the original and abstract image. The second memory means 
enhance the user* s understanding. However, locating seman- stores the abstract image as a first page of the original image. 
ticaUy significant portions of a document or among a U.S. Pat. No. 4.996 J07 to O'Malley ct al. describes a 
collection of documents, for example, and evaluating the con^)uter system that includes a capability to receive and 
relative significance of such portions can be a very arduous <5 store griq>hic images from remote facsimile machines. The 
and time-consuming task. The problem of selecting the most system includes software that can convert gr^hic images of 
significant portions erf documents and retrieving supplemen- textual material into an ASCII coded file so that cithtx 
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S^g the addressee-user the abUity to review incoming for retrieving supplemental date, 

facsimiles from a remote telephone. The system includes a These and other objects, fft-r". «fi**'f 

relay caoabiUtv the capability to print on command and to iDvcntionv^riU be apparent to ttose skilled m the art from the 

oririnatefacsimiles either from text files or scanned papers. 5 foUowing detailed descripdon. when read m conjunction 

with the acconmanying drawings and appended claims. 

SUMMARY OF THE INVENTION 

•J RRIHF DESCRIPTION OF THE DRAWINGS 

Accordingly, it is an otqect of Ihe invention to provide an BRIEI' UUiUKU-iiun ur i 

improved method and apparatus for electronic document Aprcfen-ed embodiment of the invention is illustiated m 

fsoccssing wherein supplemental data is retrieved for asso- lo accompanying drawings, in which: 

dation with the electronic document which is relevant to ia is a flow diart of a first embodiment of a method 

significant portions of the document selected without decod- ^ invention. 

ing of Ihe document FIG IB is a flow chart of a second embodiment of a 

It U another object of Ihe invention to provide a ra«*>od invention, 

and apparatus of the type described that "J^y P«««™ 2 is a block diagram of an apparatus according to the 

i^vSloSS-SO^emeth^^ 

^SJlTeS^Xti^^wliirtS^^ ^LaLe^ampleofatextdocumentc^whichselected 
S oSii^S^«r«ognition techniques toV«rieveft^ ^ have been underlinedforidentdicauon and assoaation 
t^nTAt ^ ' with supplemental infcmalion. m accordance wxd, fl.e 

B is another object of flic invention to provide a method method erabodunent of HG. IB. 
and apparatus of tfic type described that may be used to no. 4 is an example of the document of HG. 3 on which 
proviTsupplementalinf«mationrcUtingtoasourcedocu- supplemental information has been associated m accordance 
incnt to be used in reading machines for the blind. " wito the method of the mventaon. 

It is another object of the invention to provide a method deTAIUED DESCRIPTION OF THE 

and aH>aratus of the Qrpe described that may be used to PREFERRED EMBODIMENTS 

provide translatioBS for selected words in a source docu- 

In contrast to prior techmques. such as those descnbed 

Mentifl^ tased solely on an evahiation of at least one quency of linguistic terms (such as words of impoitM* 

Z'^Sll^ SuffTemeZVa related to the identified « a region of the text; font and type face va™*'""^ 

atea inerewim. ^iqi^iicmcuMu * ^ -mnhjmlTii significant words, fomiatting conventions, and so 

The moiphological image ctoaractensucs used to idenbftr J^^^^"^^i Parity, the invention will be 

significantimageunitsindudelmageumtshapedrnKsnsion^ « ^Lbed with n,ton«totte processing of a single 

typeface, font, location in the document ^^^^^^ S^tH^eS vSl be appr^tedthTt the invention 

quency of image unit occurrence. In one endKKlunent the J^^TS^f^"^'^' pcocessingTf a corpus of documents 

sieniftcant image unite are identifled accordmg to hand- is aj^ucame to ™ • 

S «aphicJ markings placed on the document by the contammg a plurahty of doeumente. 

us« Xcent word uniB of interest to the user, such as » n»e inventioD provides a lafAod and f« 
user aojaceni ^ reliieving data to supplement ttie dato ortwrt m. for example, 

enorchng « "-"^"^S' ^^^^^ f„ ^s^e^ document However, the invention is not limited 

ap^ar^s retn^^^g data to Element a document is number of sources, for sample, from internal data m 

r^e«^ The Id3s includes aVcanner for scanning the 60 another portion or page of the source document, or from 

^^il^!V^^ff^ttgmt'^^ii the docuLnt external data, such as an on-line data base. -Hie letneved 

Sr^toS; Ire^ed'for classifying date can be presented in any number ways- /or examjj^ 

^^imwe units as significant taiage uniu, and means it can be presented as margmal no es on the source 

^iS^vi^fwr^^l^l^Wl^i^nt^daterd^^ document, i.e.. the document conUining ^t^^lj" 

Sc^Vtoa^e^ts f« «S^ation with the document 65 supplemented, adjacent fte *o«rce do^^ ,C°'Scu 

^heWanTmay include a programmed digital computer whidi it pertains; jmnted as footnotes on the source docu- 

^owHe rinX seg& the doSment iTge. meat; printed on separate documents or m special formats 
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(e.g.. Braille); presented in a different fonn of display, such up to full size. The disadvantage of operating at reduced 
as a video display or synthesized voice output; or in some scale is that the word bounding boxes are only approximate; 

otho" ^jpropriate manner. howcvci» for many plications the accuracy is sufficient 

StiJl more particularly, in accordance with one aspect of The described method works fairly well for arbitrary text 
the invention <FIG. IB), selected portions of the source 5 fonts, htut in extreme cases, such as large fixed width fonts 

document can be denoted by a user by hand-marking the that have large inta-charactcr separation or small variable 

source document. Due to the varied applications for fee width fonts that have small inter-word separation, mistakes 

invention, the marking may take various forms. For can occur. The nwst robust method chooses a SE for closing 

example, a user may underline or encircle or odierwise based on a measurement of specific image characteristics, 
highlight selected words to be supplemented. An cxan^le of lo This requires adding the following two steps: 

sudi selection is illustrated in HG. 3. in which selected (1) Order die image con^nents in the original or reduced 

words in a text 7 have been marked by underlining 11. (but not closed) image in Une order, left to right and lop to 

The selection, on the other hand, may take a more bottonL 

automatic form (FIG. lA), for example, in whidi significant (2) Build a histogram of the horizontal inter-component 

portions of the source document are targeted and automati- spacing. This histogram should naturally divide into the 

cally identified by a general purpose digital computer or the small inter-character spacing and the larger intcr-word spac- 

like according to one or more naorphological image charac- ings. Then use the valley between these peaks to determine 

teristics which are predetOTuined or selected by the user. the size of SE to use for closing the image to merge 

With reference fint to FIG. 2, the method is perfomicd on ^ characters but not join wwds. 

an electronic image of an original document 5 (e.g., a bitmap Once the bounding boxes or word boxes are determined, 

image or a scanned image), which may include lines of text the locations of and spatial relationsh^ t)etween the image 

7. titles, drawings. FIG. 8. or the like, contained in one or units on a page can be determined (step 25). For example, an 

more sheets or pages of pqier 10 or other tangible fc»m.The ^glish language document image can be segmented into 

electronic document image to be processed is created in any ^ word image units based on the relative difference in spacing 

conventional manner, for example, by an input means, such between diaracters within a word and the spacing between 

as an optical scanner 12 and sensor 13 as shown, a document words. Sentence and paragr^h boundaries can be similarly 

copier machine scanner, a Braille reading machine scanner, asoeitained. Additional region segmentation image analysis 

a bitmap workstation, an electronic beam scanner or the like. can be pcrfoimed to generate a physical document structure 

Such means are well known in the art, and thus arc not ^ dcsciiptiott that divides page images into labelled regions 

described in detail herein. An output derived from, for coiresponding to auxiliary document elements lilcp- figures, 

exan^jle, a scanner sensor 13 is digitized to produce unde- tables, footnotes and die like. Figure regions can be distin- 

coded bit m^^ped image data representing the document guished from text regions based on the relative lack of image 

image f<x each page of the document, which data is stored, units arranged in a line within the region, for example. Using 

for example, in a memory 15 of a special or general purpose ^ ^ this segmentation, knowledge of how the documents being 

digital computer 16. The output from the compter 16 is processed arc arranged (e.g., left-to-iight. top-to-bottom), 

delivered to an output device 17, such as, for example, a and, optionally, other inputted information such as document 

memory or other form of storage unit; an output display 17A style, a 'Veading order** sequence for word images can also 

as shown, which may be, for instance, a CRT display; a be generated The term ^Irnage unit** is thus used herein to 

printer device 17B as shown, which may be incoxporated in ^ denote an identifiable segment of an image such as a 

a document cofaer machine or a Braille or standard form number, character, glyph, symbol word, phrase or other unit 

printer, a facsimile machine, speech synthesizer or the like. that can be reliably extracted. Advantageously, for puiposes 

The more automatic embodiment of the method of the of docinnent review and evaluation, the document image is 
invention, in which ttic image characteristics of selected segmented into sets of signs, symbols or other elements, 
image units are evaluated, wiU now be dcsoibcd with 45 ^ words, which tog^ber form a single unit of under- 
reference to FIG. lA. The first {^lase of the image processing standing. Such single units of understanding are character- 
technique of the invention involves a low level document ^ ^ image as being separated by a spacing greater than 
image analysis in whidi the document image for each page whidi separates the elements fanning a unit 
is segmented into undecoded information containing image Advantageously, a discrimination step 30 is next per- 
units (step 20) using conventional image analysis tech- 50 formed to idcntiiy die image units which have insufficient 
niques; or, in the case of text documents, preferably using infonnation content to be useful in evaluating the subject 
the bounding box method described in concurrently filed matter content of the document being processed. One pre- 
U.S. patent application Ser. No. 07/794392 filed concur- fctred method for use with text documents is to use the 
rently herewith by Huttenlocher and Hopcroft, and entitled morphologicai function or stop word detection techniques 
^'Mediod and ^paratus for Detennining Boundaries of 55 disclosed in the copending U.S. patent application Ser. No. 
Words in Text**. 07/794, 190 filed concurrently herewith by Bloomberg ct al.. 

Another method for finding word boxes is to close the &nd entitled "Detecting Function Words Without Converting 

image with a horizcmtal SE that joins characters but not A Document to Character Codes'*. 

words, followed by an operation that labels the bounding Next, in step 40, selected image units, e.g., the image units 

boxes of the connected image components (whidi in this 60 not discriminated in step 30, are evaluate<L, without decoding 

case are words). The process can be gready accelerated by the selected image units or reference to decoded image data, 

using 1 or more threshold reductions (with threshold value based on an evaluation of predetermined or scleoed image 

1), that have the effect both of reducing the inuge and of characteristics of the image units. The evaluation entails a 

closing the ^xadng between the characters. The threshold determination (stq> 41) of toe morphological image charac- 
reduction(s)aretypicaily followed by a closing with a small 65 teristics and a comparison (step 42) of the detcnnined image 

horizontal SE. The connected component labeling operation characteristics for each image unit either with the detcr- 

is also done at the reduced scale, and the results are scaled mined morphological image characteristics of the other 



07/25/2003, EAST Version: 1.04.0000 



5,748,805 

7 8 

imaee units or wife pfcdewmined morphological image rently herewith by Huttenlocher and Hopcroft. and by 

ScScT « mShological ima^ ch«acterisU^s Huttenlocher. Hopcroft and Wayner. respccuve y, a^d 

«!^^^^h7the usff entitled. rcspecUvely. "OpUcal Word Recognition By 

sciectca oy mc U5CT. , . . ... „ Examination of Word Shape." and "Method for Comparing 

One prefored method for defining the miage umt image „ ^ "t~ e 

characteristics to be evaluated is to use the word *ape ' wora bnapcs . ,. ^ ' . ^ , ri.,- 

derivation tedmiques disclosed in the copending U.S. patent Depending on the particular apphcabon. and the relative 

appUcationSer. No. 07/794391 filed concurrenflyhacwith importance of processing speed versus accuracy, for 

bv^D Huttenlocher and M. Hopooft. and entitled "A example, evahiations of different degrees of precision «inbe 

Method for Deriving Wordshapes for Subsequent Compari- performed. For example useful evaluations can be based on 
son" As described the afot^dd application, at least one. w length, width and/or other measurement dimensions of the 

one-dimensional signal characterizing the shape of the word image unit(or derived image »'>'\'^^"^?^^^''-^-^' 

unit is derived; or an image function is derived defining a the largest figure m a documen unage); the locauon of the 

boundaiy enclosing the wcid unit, and the image function is image unit in the document (mcludmg any selected figure or 

augm^d so that an edge function representing edges of paragiaph of a document unage. e.g.. headmgs^ttal 
the diaractei string detected within fee boundaiy is defined u figures, one or more paragraphs or figures), font typeface, 

over its entire domain by a single independent variable cross-section (a cross-secuon b«ng a sequence of pixels of 

within the closed boundaiy. without individually detecting similar state in an miage unu); the numba of 

and/or identifying the character or characters making up the number of descenders; the average pixd densiQr; the en^h 

dunlL of a top line contour, including peaks and troughs; the length 
* ^ . . . u ft.^ M of a base contour, including peaks and troughs; the location 

TTie determmed moiphological <;»«"^«^u^. ej the ot a ^pL^^neighboring image units; 

derived image umt shape '"P^^'f ^J- vertlcaf position; hari2on\ai inter-image unit spacing; and 

image unit are eonjpared. as noted above ("^*»>; combinid^ns of such classifiers. 

with the dcteiiiiined morphological image charactcnstic(s; t««iuuiovivu ^ ^ a f 

^d^ved image umT^ape r^sentatioos of tbc other Refenmg to FIG. IB, the embodmient of the method of 
^l^Su^ts(step4zI),^ " the inveatioDkwh^sigmfi^txnuige units arc sd^^^ 

^S^gc charart^stics to locatk specific types of based on user hand-drawa markings placedonthc document 

S^u^ (str4^^ -nie det^cd^ological does not require an 

Se diaracteri^cs of the selected image units are advan- step. Instead, the moipho ogiad mcAod "ntify^^^ 

Susly compared with eadi other fc the pun>ose of ^ i^?:*^^ ^'^'^No'T^^^ 
identifyiigeq^enceclassesofimageunitssuchthateach ^ applicaUon Ser. No. ^7/794^75 filc^con 

eSvSence dass contains most or aU of the instances of a herewith, by Bloombag, and enUtied "A Method and Appa- 

^Z ^t unit in the document and the relative frequen- ratus for Image Haini Markup I^^^<>^;.^7 ^^^^ 

des wi&which image units occur in a document can be 5^01,011 is preferably utiUzed (step 40) to identi^ 

^^ned, as is set forth more fully in the copending U.S. regions of the document image contommg^^^^ 

patent application Ser, No. 07/795,173 filed ccncuirenUy logs. THis method also permits the image units associated 

herewi^by Cass et aL now abandoned, and entiUed with the user markings to be identified. 

'•Method and Apparatus for Determining the Frequency of in instances in which multiple page documents arc 

Words in a Document with Document Image Decoding". processed, each page is processed and the data held m the 
Image units can then be classified or identified as significant ^ memory 15 (sec FIG. 1), as described above. The entirety of 

according to the ftequency (rf their occurrence, as well as the data can then be processed. 

other characteristics of the image units, such as their length. Through use of equipment such as illustrated in FIG. 2, 

For Bxan^jlc. it has been recognized that for business identified word units 11 are morphologically detected; 

communications in English, a useful combination of selec- significant morphological (structural) image charac- 

tion criteria is to select the medium fiequency word units. teristics inherent in the image form of the word units are 

It will be apiHedated diat the specification of the image detected. The non-content based image recognition aspect of 

characteristics fat tiOcs, headings, captions, linguistic crite- the invention allows image processing of documents to 

ria or other significance indicating visual features of a provide integral information about the documents wittiout 

document image can be prcdctcnnined and selected by the fost converting text in the document to character codes. Data 

user to determine the selection criteria defining a "signifi- retrieval can be then provided to automatically and direcdy 

canrimage unit Comparing the image characteristics of the access supplemental informaUon associated with the 

selected image units of the document image for matches detected word units. 

wiA the image characteristics associated with the selection ^ salient feature provicted by the method of the invention 
criteria permits the significant image units to be readily ig tj^c initial processing and identification of significant word 
identified without any document decoding. units being acconqilished widiout an accompanying require- 
Any of a number of different methods erf comparison can ment that the content of the word units be decoded. Motc 
be used. One technique that can be used, for cxanqjlc, is by particularly, to diis stage in tbc process, the actual content of 
correlating the raster images of the extracted image units the word units is not required to be specifically determined, 
using decision networks, such technique being described for Thus, for example, in such appUcations as copier machines 
characters in a Research Report entiUed "Unsupervised « or electronic i»intcrs tfiat can print or reproduce images 
Construction of Decision networks for Pattern Classifica- directly from one document to another without regard to 
tion" by Casey et al,, IBM Research Report, 1984, said aSCH ot other encoding^dccoding requirements, image 
Research Report being inctxporatcd by reference herein. units can be identified and processed using one or more 

Preferred techniques that can be used to identify equiva- morphological image chamctwistics or P^^« 

lei^cksses of wL units are the word shape comparison « i-^^units. The image "^^^ "f^^^^ 

techniques disclosed in U.S. patent ^Ucation Scr. Nos. be further optically or electromcally processed. One of the 

07/796,119 and 07/795,169 now abandoned, filed concur- advantages that results from the abiUty to perfOTu such 
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image unit processing without having to decode the image above. Or, significant key words can be automatically d^er- 

unit contents at this stage of the process is that the overall mined using the mcHphological recognition techniques 

speed of image handling and manipulation can be signifi- described above. The words thus identified as significant 

cantly increased* words or word units can then be decoded using optical 
The second phase of the document analysis according to 5 character recognition techniques, for example, for retrieval 

both method embodiments of the invention involves further of supplemental data which permits, for example. Braille 

jM-ocessing (step 50) of the identified image units in con- versions of the significant words to be printed using a 

nection with the supplemental dau retrieval. The further piastlcbascd ink printer associated with the reading 

processing can be accomplished using a number of different n^achine. Alternatively, speedi synthesized output devices 

techniques, depending upon^^ can be employed to produce a voice output representation of 

^amplc, word umU 11 (FIG. 3) that have been identified significant words as the supplemental data, 

from Ae scanned document may be decoded (step 52) by ^ ^^^^ supplemented the user may 

optical character recogmUon tediniques, which techmques „ . , . . ^ ^ ... . t. • . . * 

Z weU known in the art and thusTe not described h^in "^^.."^ f^i? 'T'' h"^;.P™"«<» 

indctaiLltedecodedwordunitsaretheDmatchedCstcpM) • «n^6»n- ni^y be ^ulueyed in a »unte of 

with associated supplemental data in a conventional mainet " 9"* "^^^ » *y«>*«=*|f« ^ ter to provide 

For instance, in embodiment, the supplemental data is "^^3* » ^ ""^"^ilZ " 

^„*.:n.-^ ■« ^ * u i« / xTTr^ * * . article entitled . . . The user would then return to the point 

contained in a data base 18 (see FIG. 2) that may contam intciesL 
specific data pertaLning to the specific decoded word imits. 

Data base 18 may be, for exan^le. a dictionary containing ^ ^^^^^^^ °^ apparatus extend this capabiUty through 

definitions of the decoded words, translations of foreign Providing the possibility of user interaction whUe the 

words, or cross-references to related documents. supplemental data is being communicated. One type of 

Alternatively, the supplemental data may be vectors or keys apparatus is a simple index marker. This can be, for instance, 

to particular data such as synthesized speech data, memory ^ * "^er dep-esses 

locations etc whenever he cr she hears a title of interest, or. for instance, 

THe supplemental data is then retrieved and outputted ^ ^^V^ ^ / 1^ (^^^ 

(step 60) to an appropriate output device. In the emboSment ^ van^ of commands. The reading 

exemplifiedinFIGS,3and4,forexample.thesupplemental n^'^<= «f onls sudi maiks of interest and retums to the 

data is translated words 45 corresponding to &e words ^"^^ ^ supplemental data is communicated, 

underlined by (he user. In this case, the translated wwds are 30 Another type of apparatus makes use of the technology of 

ou^utted by adding them to the document image in a touch-sensitive screens. Such an apparatus <^)crates by 

conventional manner so that they appear in the margin requiring the user to lay down a Braille summarization sheet 

adjacent the Une containing the words underUned by tiie user ^ hcwizontal display. The user then touches the region 

when die document image is printed or displayed, as shown interest on the screen 42 in order to trigger either a fiiU 

in FIG. 4. 35 printo"! or synthesized reading. The user would then indi- 

Thui. employing the metiiod and apparatus of the ^ate to the monitor when a new page was to be processed, 

invention, a 'translating oopieT machine, for example, may ^ ^ ^ appreciated that the method of the invention 

be constructed to assist a user in understanding documents reduces the amount of material presented to fee user for 

written in foreign languages. Depending on the degree to evaluation, and thus is capable of circumventing many 

which the user is familiar with the foreign language, tiie user 40 Problems inherent in the use of current reading technology 

may cither mark difficult ex unknown words in a printed others, such as the jMToblems associated 

copy of the document or portion thereof for translation efficient browsing of a document corpus, using syn- 

supplementation. or enter through an ^jpropriate user inter- thesized speech, and the (O-oblems created by the bulk and 

face a request that all significant words in the document or c^nse of producing Braille paper copies, and the time and 

document poitioa be automatically selected in accordance 45 required to read such copies, 

with cither predetermined or user-sdccted significance cri- 'Hie method of the invention has been described above to 

teria. The translating photocopier then ei^cr scans tiie perform document retrieval using conventional character 

marked-up copy of the document and identifies the marked recognition techniques, such as OCR. in conjunction with 

word units in accordance with the above described FIG. IB morphological identification techniques. It will be 

method embodiment, or evaluates the image characteristics 50 apf^edated, however, that direct retrieval using only image 

of selected word units in the scanned document image characteristic word unit recognition techniques may be 

pursuant to the user's request to identify significant word performed In tiie case of supplemental data which is also 

units in accordance witi) the above described FIG. lA stored as bit mapped image data compatible with the image 

embodiment The copier then retrieves the relevant transla- data of Che source document to t>e supplemented, 

tion supplemental data as described above, and pints a firesh 55 Altiiough the invention has been described and illustrated 

copy of the document or document pcHtion with translations with a certain degree of particularity, it is understood diat the 

of the underlined words in tiie mai^;ins opposite the under- present disdosure has been made only by way of cxan^e, 

lined words, as shown in FIG. 4. and that numerous changes in the combination and arrange- 

Another plication of the doctiment supplementation meat of parts can be resorted to by those skilled in the art 

techniques of the invention is in reading machines for the 60 without departing from the spirit and scope of the invention, 

blind. One embodiment supports the designation by a user of as hereinafter claimed, 

key words, for example, on a key word list to designate We claim: 

likely points of interest in a document Using the user 1. A method for electroiucally processing an electronic 

designated key words, occurrences of the wcxd can l>e found document image, comprising: 

in the document of interest by OCR techniques or the like, 65 segmenting the document image into word units without 

and regions of text forward and behind the key word can be decoding the document image, each word unit coire^ 

retrieved and processed using the techniques described sponding to a word in said document image; 
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identifying significant ones of said word units in accor- 
dance with at least one selected mocphological image 
characteristic of said word units without dctennimng 
their content other than said at least one selected 
mcHphological image characteristic; and 5 

retrieving supplemental data related to the identified sig- 
nificant word units. 

2. The method of claim 1 wherein said step of identifying 
significant word units includes a step of classifying said 
word units according to frequency of occunence. 

3. The mctfiod of claim 1 wherein said step of identilying 
significant word units includes a step of classifying said 
woffd units according to location within the document Image. 

4. TTic method of claim 1 wherein said at least one 
selected mctphological image characteristic incUides image 
characteristics defining word units having predetermined 
linguistic criteria. 

5. The method of claim 1 wherein said at least one 
selected morphological image characteristic includes at least 
one of a word unit shape dimension, font, typeface, number 
of ascender elements, number of descender elements, pixel 
density, pixel cross-sectional diaraderistic, the location of 
word units with respect to neighboring word units, vertical 
position, horizontal inter-image unit spacing, and contour 
characteristic of said word units. . . . 

6. The method of claim 1 wherein said step of identifying 
significant word units comprises identifying word units 
having an associated hand-drawn marking created by a user, 

7. The method of daim 1 wherein said step of retrieving 
supplemental data comprises retrieving foreign language ^ 
data corresponding to said identified significant word units. 

8. The method of claim 1 wherein said step of retrieving 
supplemental data comprises retrieving data from within the 
document 

9. The method of claim 1 wherein said step of retrieving " 
supplemental data comprises retrieving data external to the 
document 

10. The method of claim 1 further com{nising modifying 
the document image with retrieved data to jm>vidc a docu- 

40 

ment annotation. 

11. The method of claim 10 wherein the document anno- 
tation is in the form of marginal notes which are located in 
a noargin of said modified document image. 

IZ. The method of claim 1 wherein said step of retrieving 
supplemental data retrieves Braille versions of the identified 
significant word units. 

13. The m^od of claim 12 ftuther comprising outputting 
said retrieved Braille versions of die identified significant 
word units in printed form. 

14. The method of claim 1 wherein said step of retrieving ^ 
supplemental data for the document comprises retrieving 
syn&esized speech versions of the identified significant 
word units. 

15. A metiiod for translating a selected word in a text 
document cosi^nising: 

marking the selected word in the document text with a 

hand-drawn graphical notation; 
scanning the text document to produce an undecoded 

scanned document image; ^ 
segmenting the document image into inoage segments 

without decoding of the document inoage; 
evaluating the morphological image characteristics of the 

scanned document image to identify said graphical 

notation; 65 
identifying the image unit associated with said identified 

gr^hical notation; 



retrieving translation data related to said identified image 
unit; and 

incorporating said retrieved translation data in said 
scanned document image. 

16. The mcdiod of daim IS wherein said retrieving step 
indudcs decoding the identified image unit and matching the 
decoded image unit with decoded data entries in a dictionary 
database. 

17. A method for dcclronically processing an undecoded 
dooiment image containing word text conoprising: 

segmenting the document image into word image units 
without decoding the document image, each word 
image unit corresponding to a word in said document 
image; 

evaluating selected word image units according to at least 
one morphological image characteristic of the sdectcd 
word image units without determining their content 
other than said at least one morphological image char- 
acteristic to identify significant word image units; 
retrieving supplemental data rdated to die identified sig- 
nificant wcffd image units; and 
outputting said retrieved supplemental data, 

18. An ^araius for retrieving data to supplement a 
document comprising: 

means for inputting the document to produce an unde- 
coded document image; 
means for segmenting the document image into word 
units having undecoded information content without 
decoding the document image, each word unit corre- 
sponding to a word in said document image; 
means for evaluating selected word units according to at 
least one morphological image characteristic of said 
sdectcd word units, without determining their content 
other than said at least one morphological image 
djaracteristic* to identify significant word units; 
means for retrieving supplemental data related to the 

identified significant word units; and 
an ou^t device whidi utilizes the retrieved supplemental 
data. 

19. The apparatus of daim 18 wherein said means for 
segmenting the document image, said means for identifying 
significant word units, and said means for retrieving supple- 
mental data comprise a progKHmmed digital computer. 

20. The apparatus of daim 18, wherein the means for 
inputting the document conqxiscs a bitmap w^kstation. 

21. The apparams of claim 20 wherein said retrieved 
supplemental data is translation data. 

22. The method of daim 1. wherein said step of retrieving 
supplemental data comprises retrieving data fi:om an online 
database. 

23. The apparatus of claim 22 wherein said output device 
is a Braille printer for producing document copies in Braille 
format 

24. The ^aratus of claim 22 wherein said output device 
is a speech synthesizer for producing synthesized speech 
output corresponding to said identified significant word 
units. 

25. An ^aratus for retrieving data to supplement a 
document ooiiyMising; 

a copier machine having a scanner for inputting the 
document to produce an undecoded document image; 
means for segmenting the document image into word 
units having undecoded Information content without 
decoding the docimieat image, each word unit corre- 
sponding to a word in said document image; 
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means fear evaluating selected word units according to at 
least one moq^hological image characteristic of said 
selected word units, without determining their content 
other than said at least one moiphoiogical image 
diaractcristic. to identify significant word units; 

means for retrieviog supplemental data related to the 
identified significant wofd units; 

said copier machine including an ou^ut device for pro- 
ducing printed documents; and 

means for incofporating said retrieved supplemental data 
into the document image for printing as part of a printed 
document by said output device. 

26. An apparatus for retrieving data to supplement a 
document comprising: 

a Braille reading machine including a means for inputting 
the document to produce an undecoded document 
image; 

means f<a' segmenting the document image into word 
units having undecoded informatioQ content without 
decoding the document image, eadi word unit corre- 
sponding to a word in said document image; 

means for evaluating selected word units according to at 
least one morphological image diaracteristic of said 
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selected word units^ without determining their content 
other than said at least one morphological image 
characteristic, to identify significant word imits; 

means for retrieving supplemental data related to the 
5 identified significant word units; and 

said Braille reading machine including an ou^ut device 
which utilizes the retrieved supplemental data to pro- 
duce an ou^ut intelligible to a blind user. 

27. The method of claim 1. further comprising printing 
10 said retrieved supplemental data with a copier machine. 

28. The method of claim 27, wherein, prior to performing 
said segmenting step, said electronic document image is 
input to said copier machine. 

29. The method of claim 28, wherein said electronic 
1^ document image is input with a scanner of said copier 

machine. 

30. The method of claim 15. wherein said text document 
is scanned using a scanner of a copier machine. 

31. The method of claim 30. further comiMising printing 
^ the scanned document image incofporating said retrieved 

translation data with a printing device of said copier 
machine. 

***** 
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ABSTRACT 



Recorded data is read by an inaa^ sensor from a recording 
medium on whidi the data is two^imcnsionally recorded in 
a mesh pattern, and the read data is stored in an image RAM. 
The stored data is subjected to data decoding and error 
correction to reproduce target data. A scaiming reference 
pattern of a mesh pattern recorded on the recording medium 
serves as a guide when the stored recorded data read by the 
image sensor is reproduced by scanning in the image RAM. 
A reproduction apparatus has a capability of coping with a 
partial dcstructioii of the scanning reference pattern. Ba<x 
checking codes ar e added to the rec ording medium in 
addition to the target dAta. T hesettems or data are recorded 
as an cncQ j died image aftcr~pemg subjecteo to scrambling 
processing and randoniizationr^ rcproductton 
apparatus can easily cope with an error when the encoded 
image is reproduced. 

2 Claims, 32 Drawing Sheets 
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METHOD AND APPARATUS FOR 
RECORDING/REPRODUCING MESH 
PATTERN DATA 

This is a division of application Ser. No. 07/S69,012 filed 
Apr. 14. 1992 now U.S. Pat No. 5,454,054, which is a 
Division of Set. No. 07/530,630 filed May 30, 1990 (now 
U.S. Pat No. 5.153,928). 

BACKGROUND OF THE INVENTION 

1. Field of Che Invention 

The present invention relates to a data recording/ 
reproduction system for encoding binary data to record same 
as an image, and for reading a recorded image and decoding 
the read image into binary data. 

2. Description of the Related Ait 

A bar code technique is known as a technique for reading 
an encoded image from a recording medium (recording 
sheet) to reproduce binary data. 

For exaiiq)le, Pat Nos. 4,422361, 4,437378. and 
4,464,966 disclose this technique. However, it is difficult to 
increase a recording density of a bar code in terms of its 
structure, and the bar code is not suitable for handling a large 
volume of data. 

Unexamined Japanese Patent Application Publication No. 
53-73026 discloses a technique wha-ein an image in which 
some meshes of an ixJ (e.g., 3x3) matrix are set to be black 
meshes, and the remaining meshes are set to be white 
meshes; is read to recognize a black^d-white mesh pattern. 
The use of the matrix encoded image can easily inaease a 
data volume by increasing ^e number of meshes-included in 
the matrix. Another technique using a similar matrix image 
is also described in Magazine I/O, May, 198S, pp. 121 to 
125, "Dan^ List Read by Image Scanner**. U.S. patent 
^pUcation Ser. No. 07/389^87 (filing date: Aug. 3, 1989, 
inventors: Morikawa et al. now U.S. Pt. No. 5,042,079, 
assigned to the present assignee proposes the following 
technique. That is, a mesh pattern obtained by encoding data 
by black-and-white meshes selectively formed on a mesh 
matrix consisting of a large number of meshes is used as a 
data body of an encoded image in place of a bar code, and 
such an encoded image is read and decoded. 

According to the invention disclosed in this U.S. Ser. No. 
07389,287, since data bits are expressed by a black-and- 
white pattern of meshes which are two-dimensionally 
arrayed at small intervals, a recording density can be greatly 

However, when an encoded image itself <hi a recording 
medium is stained, e.g., contaminated, and a scarming ref- 
erence pattern is partially deformed, omitted, or damaged 
thereby, the above-mentioned decoding method may erro- 
neously recognize the contamination as a segment of the 
scanning reference pattern, and position errors of meshes of 
the mesh pattern on image data which are determined on the 
basis of the recognition result occm*, resulting in a wrong 
black-and-white recognition result For example, if a certain 
white clock mesh (hereinafter referred to as a dock) of a 
syndut>nous pattern as a sut>-scanning reference pattern is 
stained in black and becomes continuous with black clocks 
before and after this white clock, when clocks of the 
synchronous pattern arc delected on image data, the white 
clock is not read (i.e., three, i.e.. black. White, and black 
clocks are detected as one black clock), and this error causes 
a decoding error of all other clocks. In this state, even if an 
error is detected later in error correction processing using a 
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checking code (redundant code included in a mesh pattern), 
conection is iiiq>ossible to achieve since the error is con- 
siderably beyond a correction capability. 
A position error of a guide line as a main scanning 
5 reference caused by contamination influences a position as 
a central position of meshes in the vertical direction thereof 
obtained by equally dividing a line connecting correspond- 
ing positions of two guide lines on a scanning line image. As 
a result, a noncorrectablc sampling error occurs later. 

A recognition error of image data may occur due to 
various other causes. For example, the causes include a 
defect (caused by poor printing precision, a variation in 
density, or the like) of an encoded image on a recording 
mediumi, problems of an image sensor (an image resolution 
of the image sensor, an image distortion in image processing 
by the image sensor (distortion of a lens, a change in 
scanning speed or direction), luminance stability of a lig^t 
source, a threshold level f(H- binarizing analog Image data, 
and photoelectric conversion characteristics of an image 
20 sensor such as a CCD), and ttie like. 

Most of such errors are caused by the fact that a mesh 
pattern is adopted to increase a recording density on a 
recording medium. In other words, most of errors are related 
to the fact &at an encoded image on a recording medium is 
^ constituted by a two-dimensional array of image elements 
encoding information units in order to effectively use a 
two-dimensional space on the reccKding mediimt 

SUMMARY OF THE INVE^mON 

It is therefore an object of the present invention to provide 
a data reading method and apparatus, which can detect 
correct positions of scanning reference patterns on image 
data even when scanning reference patterns of an encoded 
2j image having a mesh pattern as a data body are partially 
destroyed by. e.g.. contamination on a recording mediunL 
anoth er object of die present invention to provide a 
r ecord in g mgdiumj or recording a n enco^ea i mage, w mch 
Can'eg^gvely s^e the above-mention5i er ror problems 
^ while maintai ning a relatively high rccoromg oensiQr , to 
t^rWide a data recording method and apparatos to r record ing 
An encoded image on die re cording medium, and to ^ovidc 
a^iatajcpr oductton metnoa and a pparatus ca pable of rcpro-^ 
diign gj esir ed da ia^free firoin an cnor or at a low error r at€_ 
43 from (he enco god~ima gc. 

According to oneaspecTof the present invention, irr gpe . 
data r e presenting anjmage ig read fr omTre c ording in e^Sa( 
whjcH" recoras^ *^ 'Tl^ff'* ^nc^udin g a mesh pattern and a 
main scanning reference pattern, and the read image data i s 
50 divided into.a plurality o f partial image data segm ents. F or 
each partial image data segiisjentr'l~feat\ire pixeP^oup 
featining a piece of the main scanning reference pattern 
included in each partial image data segment is searched from 
the partial image data piece- F or die partial Jma^e d ata 
55 s egment from which no feature pixel group i s detected up on 
se^5iinE, ttic position of the piece o^ ihe mam scanning 
reference pattern in that partial im^e data serene " is 
dcicrmthed from the position of apicceof the inain scann ing 
r efcrcnccn^ttcrn represented by t he feature pixel"?ou p 
$0 detected*itipon searcm ng or another parti al image date se g- 
ment-33 fe-g3si tion d a ta Of pfg^T^lhe mak scan ninjg 
r eference pa tte rn in die ima ge d ata are gene ra teOnd black 
and white rfe shc s of thTmes h ixi ttem of the "image data is 
recognized on the basis of the gen erated positioh^data. 
65 ' According to tks aspect, even when the main scanning 
reference pattern on the recording medium is partially 
destroyed by. e.g.. contamination, the destroyed portion is 
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limited to a local area, i.e., a partial image data piece, and tfie medium which records the image induding a mesh patteni 

unimown position of the main scanning reference pattern in and a scanning reference pattern having a synchronous mark 

this local area can l>e determined by positions rei»%sented by synchronous with the mesh pattern. The synchronous mark 

feature pixel groups obtained from o&cr normal main scan- of the scanning reference pattern is searched from the read 
ning reference pattern segments. The positions of the main 5 image data. During this searching, at least one-synchronous 

scanning reference pattern before destruction can be restored I5 detected to determine feature parameters including 

with high precision. Therefore, black and white meshes of jts position. The position of an adjacent synchronous mark 

tfie mesh pattern can be reliably recogmzed with reference dctennined feature parameters. The 

to tiie posmons of the mam scanmng reference pattern, ^^^^^^ synchronous maik is actually mekTured with ref- 

According to one preferred aspect of the present ercacc to the predicted position. The position of the actual 

kvention, the mam scanmng reference pattern consist of measurement Jesuit is compared with (he predicted posi^^ 

two contmuous lines (guide lines) extending on both sides of « • i* 11 * 

a mesh pattern. As a pixel group featutini a piece of this ^l**^" comparison result essentially represents a 

guide linVthree, c.g.. white, black, and white run lengths (to f ^incidence, the position of the adjacent synchronous mark 

be referred to as a guide line set hereinafter) which are dctctmmed in accordance with the actual measurement 

arrayed in a depth direction of image daU (in a direction However, when the con^son result essentiaUy 

perpendicular to a main scanning line image) are used. The represents a nonooincidence, the position of the adjacent 

guide line set is searched on partial image data segments of synchronous marie is determined on the basis of the dctcr- 

each of the two guide lines. The position of one of the guide mined feature parameters. The prediction, actual 

line pcxtions (often both guide Line portions)^ for which measurement comparison, and determination operations arc 
search is failed is obtained by interpolation from the normal 20 repeated for all the synchronous marks, thereby generating 

(search is successful) guide line positions before and after position data of all the synchronous marks. Blade and white 

this line portion. Sudi two guide lines can cope with a meshes of the mesh pattern are identilied based on the 

change in scaiming direction caused when an image sensor generated position data. 

is manually scanned. Since the pattern itself is an elongate d According to this aspect, a portion of the scanning refer- 
continuous line and has a lefftuy^i wMd^ is not pres;^iii 25 cnce pattern, whidi is destroyed by, e.g.. contamination can 

other mesh image elem ents it is advantageo us to recogm- ^e detected since the predicted point obtained in the 

tion. rUtlnen2iQa*e, even when imaees, such as characters, _^ 1 ^ f 

WBa-are not preterable for dec^B. are ananged af SSd ^^'^"'^ f^f measurement, and compaiKon operations 

a> encoded image, the guide lines c£; serve as boundari ^ with die actuaUy measured pouU. In Uus 

f or an encoded Snaee ~ — "i* destroyed synchronous mark is 

-n.e present inveati ^ can be applied to a main scanning '° on alr«^ detennined feature parametcn 

J. ju* t • -l'TlTm • (e.g., an interval or position of the synchronous mark 

reference psfttem havmg an arbitranly appropnate pattern. f . i *i.j-_j_^-vr«^ 

^ * - ui J obtained from a mark near the destroyed portion). Therefore, 

The following patterns are more preferable since recogmtion ^ ^ . ^ ^ . n 

crfthemains^LinBtefcrcnceii^canbefacilittSd:(a) 7" ^f^'g reference pattern is partiaUy 

^ . . * , f ^ / * * -J destroyed, original synchronous mark positions can be 

a pattern having a positional feature (e.g., present outside a • jt --i. r^V . . r«L li i j l*^ l 

in«hpattcm)T(b)a Mttan which is uniqtc by itscJf. and can °^<=d «^ ^ picasion Thus black and white meshes 

be eag^iish^ f^m meshes J a ^ pattern. t'^^^^^^'^ ^ 

A data Start mark may be formed at a position ( before a „ , ^ , . ^ ^ 

m5h7S55 sSSe d Moi to a mesh pa^ durj^ jT^ scamimg i^cr^ce pattern may be one or both for 

n ing an encoded imag £jaLAiin&-iiaag&-scnsar>^ and a dat a 40 scanmng and sub-scanning, 

c ad mark may be formed at a position scanned aftg the When a scanaiog direction or a scanning speed is varied 

mesh pattCTn. In this case, when these data start and end during scanning of an encoded image by an image sensor, if 

marks arc detected from image data during a decoding position of a synchronous mark is detcimincd on the 

operation, more reliable data decoding is assured. For basis of an actual measurement result the feature parameters 
exan4)lc. when an end portion of a main scanning reference 45 serving as a standard fOT the next synchronous mark arc 

pattern (e.g., the above-mentioned guide line) is destroyed, preferably iqxlated by Uie actual measurement result 

if an encoded image format is detcimiDed such that the data The meaning of a **positioD^ used when an actually 

start and end marks are located on the main scanning line or measured position and a predicted position are compared 

a line group crossing the end portion of the main scanning should have a broad sensew and the ^sition*" can indude a 
reference pattern, the destroyed end portion of the main 50 parameter whose value is changed in assodation with 

scanning reference pattern can be detennined from a line of destruction. For exanople, an interval between adjacent syn- 

the main scanning line image from which a piece of the data chronous clocks is varied when the central position of one 

start or end mark is detected and a line estimated from synchronous mark is considerably dianged by contamina- 

detection results of undestroyed main scanning reference tion. Thus, a predicted synchronous clock interval does not 
pattern portions. 5j coindde with an actually measured synchronous clock inter- 

'Hie size (depth) of partial image data as a unit area fa: val. As for the size of a synchronous mark, a destroyed 

searching a main scanning reference pattern piece or the size synchronous mark can be detected by comparison since its 

of a piece to be searched may be localized according to a size is considerably different from a predicted size of a 

searching result and may be reduced stepwise until it is regular synchronous mark. 

almost equal to, e.g.. the size of a destroyed piece. For 60 When a moving direction (sub-scanning direction) of a 

example, when searching is unsuccessful in two continuous line image sensor for scanning an encoded image is varied 

partial image data, a piece to be searched is halved to retry during scanning, it is preferable that at least two arrays of 

searching. In this manner, the position of the main scanning synchronous marks arranged along an average sub- scanning 

reference pattern can be determined with still higher prcd- direction of an encoded image are prepared, and are 
sion. 65 arranged to be s^arated in the main scanning direction. 

ng to another aspect of the present Invention, When the two arrays of syndironous marks are used, the 

image data representing an image is read from a recording synchronous marks in the two arrays of syndironous marks 
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can have a onc-to-onc corrcspoadcncc with each odicr. 
Iherefore, by utilizing this nature, a position of a portion 
where a synchronous mark in one array is destroyed can be 
determined with high precision OD the basis of an actually 
measured position of a coircsponding synchronous mark in 
the odier array, as well as on the basis of the relationship 
(position vector) between actually measured positions of 
adjacent normal synchronous maiks in the two arrays of 
synchronous marks (since an interval between adjacent 
syndironous marks on each array can be minixnized, the 
change is very small even if the scanning direction is 
dianged at the interval). 

According to another aspect of the present invention, in 
order to record, on a recording medium, aa encoded image 
in which Image elements e^essing elements of data are 
arranged at two-dimensionally distributed positions on the 
recording medium, i.e.^ an encoded image constituted by a 
two-dimensional matrix of image elements representing data 
units, an error checking code is added to a iirst or original 
two-dimensional (data) matrix constituting target data to 
form a second two-dimensional matrix, and an encoded 
image is printed on the recording medium according to the 
formed second two-dimensional matrix. 

Therefore, since an encoded image on the recording 
medium includes an image dement group representing an 
OTor cheddng code in addition to target data which is 
significant as original data, the error checking code can be 
effectively used during data reproduction. 

More specifically, in a data reproduction method and 
^aratus according to this other aspect of the present 
invention, the above-mentioned encoded image is read £rom 
a recording medium as image data> image elements are 
found from the read image data, data elements expressed by 
the image dements arc obtained to form a two-dimensional 
data matrix, and an error on tiie matrix is corrected using an 
error checking code induded in this matrix, thereby r^ro- 
ducing target data correctly. 

FurtheriDore, in a data recording method and apparatus 
according to still another a^ct of the present invention, a 
means for taking a countermeasure against a burst error is 
added to the data recording method and apparatus described 
earlier. The order of dements of a two-dimensional data 
matrix is changed so that continuous data dements on die 
two-dimensional data matrix obtained by adding an czror 
cheddng code to target data are arranged as scattered image 
dements on a recording medium. An encoded image is 
recorded according to the two-dimensional data matrix in 
whid) the order is changed, so that a two-dimensional data 
matrix whose order coinddes with that of the two- 
dimensional matrix of image elements on the recording 
medium is detenuiiied as one not before ^e order change but 
after the order change. 

When the order of the matrix is changed (scrambled or 
interleaved), even if a contamination as a cause of a burst 
error is attached onto the recording medium, this portion 
oorrespoods to discontinuous data elements on a data matrix 
before the order change. Therefore, a burst error on image 
data can be removed during data reproduction. 

More specifically, in a data rq)roduction method and 
^)paratu5 according to still ancther aspect of the present 
invention, image dements are found from read image data 
to fonn a two-dimensiODal data matrix corre^nding to a 
data matrix whose order is changed, as descTit>ed above, Le., 
a data matrix whose order coinddes with the order of image 
elements on the recording medium. Thereafter, the order of 
the matrix is restored (descrambled or ddnterleavcd) to an 
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original c»-der. thereby forming a two-dimensional data 
matrix corre^nding to a data matrix obtained when an 
error checking code is added to target data. An error on this 
matrix is corrected based on the error checking code 

5 induded in the matrix, thereby repiodudng the target data. 
As a result, even if a burst error is included in a data 
matrix when the data matrix whose order coinddes with the 
order of elements of an encoded image on the recording 
medium is obtained, the error appears at separate locations 

to on a descrambled data matrix loduding the error checking 
code. Therefore, the error can be easily corrected based on 
the error diecldng code. 

Furthermore, in a data recording method and apparatus 
according to a still further aspect of the present invention, 
when an encoded image constituted by a two-dimensional 
matrix of image elements which express data dements by 
black and white meshes is recorded on a recording medium, 
the values of elements on a first or original two-dimensional 
data matrix ai% changed according to a pseudo random 
number to form a second two-dimensional matrix 
(randomized data matrix) so that data elements successively 
having the same value on the first two-dimensional data 
matrix constituting given data are not recorded as image 
elements successively having the same density on the 

^ recording medium. The encoded image is printed on a 
recording medium in acccardaoce with the formed second 
two-dimensional matrix. 

Therefore, image elements having the same density 

^ hardly continuously appear on the recording medium, and a 
set of image elements, densities or gray levels of which are 
irregularly changed is obtained. This irregularity is advan- 
tageous for characteristics of an optical image sensor when 
an encoded image is read by a line image sensor, and can 
prevent an caxor caused by a change in photoelectric con- 
version diaracteristics of die image sensor caused by long 
portions having the same density. 

In a data reproduction method and ap{>aratu5 according to 
a still further aspect of the present invention, which reads 

40 such an encoded image, image elements are found from read 
encoded image data to form a randomized data matrix 
having data element values according to densities of the 
image elements, and thereafter, the values of the elements of 
the matrix are changed according to a predetermined pseudo 

45 random number (derandomized) to obtain a data matrix 
before randomizatioa, thereby reproducing data. 

BRIEF DESOUPnON OF THE DRAWINGS 

Other objects and features of the present invention will be 
50 understood for those skilled in the art from the description 
of the preferred embodiments of the present invention taken 
in conjunction with the accompanying drawings, in which: 

FIG. 1 is a block diagram of a data reading apparatus 
according to die first embodiment of the present invention; 

FlO. 2 exemplifies an encoded data on a recording 
medium as an object to be read; 

FIG. 3 exenoyplifies an Image obtained by scanning die 
encoded image shown in FIG. 2 by a manual image scanner; 
60 FIGS. 4A and 4B show a flow chart for storing image data 
obtained by scanning an encoded image, and recognizing a 
guideline as a main scanning reference pattern from die 
stored image data to perform main-scanning decoding; 

FIGS. 5A and 5B show a flow chart for recognizing a 
65 synchronous mark array as a sub-scanning reference pattern 
from the main-scanning decoded image data to perform 
sub-scanning decoding; 
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FIG. 6 shows a memory of an image RAM; FIG. 33 shows a bit matrix from which a DC con^wnent 

FIGS, la) and 7(fr) arc views for explaining a guide line is removed, Lc., a final hit matrix; 

set featuring a piece of a guide line; FIG. 34 i$ a flow chart showing the overall operation of 

no. 8 shows an encoded image around which characters « r^oduction apparatus; 

arc present, and a jduraUty of search blocks obtained by ^ HG. 35 shows a symbol matrix of data with an crFor 

dividing the encoded image; checking code after- symbol dcscrambling during data repro- 

FIG. 9 exen^lifies an image including a contaminated Auction together with P and Q vectors processed in error 

guide line and a synchronous mark anay; correction processing; 

FIG. It shows main-scanning decoded image dato cor- iq ^ ov«aU flow chart of error correction 

responding to the image shown in FIG- 9; processmg; 

HG. 11 is a Wock diagram of a data reading apparatus , ^ ^ ^f^^. ^ °^ 

according to the second embodiment of the present invcn- processmg shown m HG. 36; 

^qq; fig. 38 is a detailed flow diart of erasure processing 

FIG. 12 shows an encoded image having no data end 15 shown in FIG. 37; and 

oiai^ FIGS. 39{a) -39(g) show encoded images with eirors 

HG. 13 is a schematic view of an encoded image with T^Jf^.^ ^rT^'wn? reproduction 

three guide lines; apparatus of die third embodmient 

FIG. 14 is a schematic view of an encoded image widi one DETAILED DESCRHTION OF THE 

guide line; PREFERRED EMBODIMENTS 

FIG. 15 is a schematic view of an encoded image wifli one Preferred embodiments of die present invention will be 

syndironous mark array; described hereinafter with reference to flic accompanying 

FIG. 16 is a schematic view of an encoded image with drawings, 

syndironous mark arrays in which elongated black marks 25 First Embodiment 

are arrayed as clocks; FIG. 1 shows the overall airangement of a data reading 

FIG. 17 is a schematic view of an encoded image widi apparatus 10 according to tiie first embodiment The object 

synchronous mark arrays having different cycles ; ^ apparattis is to read a mesh-pattern image 20 (FIG. 2) 

HG. 1» is a view for explaining processing for identifying ^'^ * recording mediuin such as a papex sheet using 

black and white meshes by a middle point method from the ^ ^ ^ ^ ^^"^ 

center of four clocks of the synchronous mark arrays; E} , . 

, ^. . . The image sensor 11 comprises a manual line imace 

FIG. 19 shows an arrangement of image data on an image :„«i,T^j-« - r^r^ : 

« - . , J. . .« ™ J IJx ±. . sensor including a CCD sensor element array. The uuage 

RAM mcludinc imddlc points CI and 02 shown m FIG. 18: ^* . n ^ s i^ • 1/. 

1 111 1 1.5 ^ ^Mou I vwu uMiMKj a, sensor U is manually moved in a sub-scamung direction 

FIG. 20 shows the relationship between four positions of 35 (lateral direction) of the recording medium to scan the 

a sampling reference pattern and a mesh position located encoded image 20 on the recording medium, thereby gen- 

inside a rectangle defined by die four positions on read crating concsponding image data. More gxtcificaUy, the 

image data; sensor elements of the sensor array photoeledrically convert 

FIG. 21 shows the relationship between four positions of input light ic., input reflected lig^t according to a black- 

a sampling reference pattcm and a mesh position located 40 and-white pattern of pixels on the recording medium facing 

inside a rectangle defined by the fcAir positions on a record- the sensor elements into analog signals. Analog image 

ing medium in correspondence with FIG. 20; ou^uts of die sensor elements are sequentially and cydi- 

FIGS. 22(a) -22(/) views for explaining iMX)cessing for dcnvcd through an analog mult^>lcxcr or the like, so 

coordinate-converting the central coordinates of meshes on that a Unc image ou^ut for all the sensor e l ements of the 

a reo^ding medium into the central positiOD of meshes on ^5 sensor array can be obtained for each predetermined period 

image data; (main scanning period) detmnining a time for one main 

FIG. 23 is a block diagram showing a data recording Thereafter, the line image ou^t is binariz^d 

apparatus according to the third embodiment of the present *>y « arcutt (e.g., a black pixd="i; and a white 

invention- pixel=*X) ), and fee binary senal unage data is supphed to a 

TTir^ a 1- u • *!. 11 *T r ^ controller 12 together with a timing control signal. The 

^ IS- overaU operation of ^^^^^^ conLls the image sensor U and its ^^heral 

the data recordmg apparatus shown m FIG. 23; ^^^^^ conversion of the input serialLige 

HG. 25 shows a bit matrix of target data; j^ta into paraUel data. 

FIG. 26 shows a symbol matrix of the target data; A CPU 13 operates according to a program stored in a 

FIG. 27 shows a matrix In which an error checking code 55 ROM 14. During scanning of an image by the image sensor 

is added to the target data; 11 < ^c CPU 13 writes parallel data in an image RAM 15 

FIGS. 28A and 28B show a matrix after a first scrambling ^^*=^ ^ ^ paraUcl-converted by the 

operation of a data matrix with tiie cnor checking code; controUer 12 Upon completion of the image scanning, the 

nGS.29A and 29B show a matrix after a second scram- ^ ^ * ?^ 
hline ooeration' 60 in the image RAM 15, and records the decoding result m a 
—icT^ . * , . . ^ . . RAM 16. The decoding operatioii can be roughly classified 
FIGS . 30A and 3«B show a matrix after a third scrambhng processing foi recognizing a sampling reference pattmi 
^P*^^**"' of data from the image data, and processing for identifying 
HG, 31 shows a scrambled bit matrix; black and white meshes of a mesh pattern as a data body in 
FIG. 32 shows an arrangement of a pseudo random 65 accordance with the recognition result. According to ttiis 
number generator for removing a DC component from a embodiment, even if the sampling reference pattern is par- 
scrambled bit matrix; . tially destroyed by, e.g., contamination, the respective posi- 
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tioDs of the saic^lingrefereacr pattern on the linage data can encoded image format shown in FIG. 2, a straight line 

be determined high precision, and an accurate decoding connecting the centers of diese corresponding clocks passes 

result of the mesh pattern can be obtained. This will be the center in the lateral direction (sub-scanning direction) of 

described in detail later. meshes in a certain column of the mesh pattern 22. 

FIG. 2 shows the encoded image recorded on the record- 5 Therefore, the positions of clocks in the pair of synchronous 
ing medium as an object to be read by the data reading mark airays 25 are detected from the image data, thus 
apparatus 10. The encoded image 20 shown in FIG. 2 detennining the sub-scanning central position or a sub- 
includes a mesh pattern 22 as a data body. The mesh pattern scanning mesh range of meshes in the mesh pattern 22 on the 
22 consists of a large number of meshes arranged in a image data. 

matrix. Each mesh is selectively set in black or white to lO In the encoded image 20 shown in FIG, 2, a data start 

express (encode) a bit as a unit of data. F<H- example, a black mark 23 consisting of a checkertK>ard mesh pattern is 

mesh C3qjresses a bit and a white mesh expresses a bit formed in an area scanned i»ior to the mesh pattern 22. A 

'*0". In FIG. 2. the mesh pattern 22 has 48x72 bit informa- data end mark 24 in which laterally elongated black and 

tion. Furtiicrmore, the encoded image 20 has patterns or white stripes alternately ^)pear along the main scanning 

marks for indicating main scanning and sub-scanning data is direction is formed in an area scanned after scanning of the 

sampling references for the mesh pattern 22 as a data body. mesh pattern 22. Thus, since the data start and end marks. 23 

In FIG. 2. the main scanning and sub-scanning sampling and 24 cony)rise periodic a regular patterns, they can be 

reference marks arc separately formed. The main scanning easily detected on the image data. Since the data start and 

sampling rcfcrcnce is defined by two guide lines 21 extend- end marks 23 and 24 respectively con^se patterns which 

ing in the sub^scanning direction along the upper and lower 20 can be easily visually distinguished from each other, a user 

sides of the mesh pattern. The sub-scanning sa]x:^ling ref- can easily determine the scanning direction (moving 

erence is defined by two synchronous mark arrays 25 each direction) of the image sensor 11 with respect to the encoded 

of which extends along the inside of the corresponding guide image 20. Even when an end portion of the guide line 21 as 

line 21 and has repetitive patterns of alternate black and the main scanning reference is destroyed by, e.g., 

white.An interval oar pitch between adjacent black and white 2S contamination, start or end of data can be detected upon 

meshes in each syndironous mark array 25 is syndtronous detection of the data start or end mark 23 or 24, and the 

with, Le., coincides with a sub-scanning interval of meshes destroyed position of tiie guide line 21 can be predicted or 

in the mesh pattern 22. Thus, each mesh of the syndironous evaluated. 

mark array 25 will be called a "clock" hereinafter. In FIG. 2, W1=W2=W4=W5. Where Wl represents a 
The reason why the two guide lines 21 are formed as the 30 distance between a start end of the guide lines 21 and a start 
main scanning reference patterns along ttie moving direction end of the data start mark 23, W2 a width of the mark 23, 
of ttie image senscr 11, icthe sub-scanning direction of the W4 a widtfi of the data end mark 24, and W5 a distance 
image 20 to be separated at a predetermined distance, is to between an end terminal of the mark 24 and ends of the 
cope with a case wherein the moving direction of toe image guide lines 21. W3 designates the length of the data body, 
sensor 11 is changed during manual, scanning of an image. 35 FLG. 3 shows an image obtained when die encoded image 
More specifically, image data from the image sensor 11 is shown in FIG. 2 is manually scanned by the image sensor 11 
Stored in the image RAM 15 to be identified from each other relatively carefully. Even when the sampUng reference pat- 
in units of main scanning lines. A direction of a line image tern is partially omitted, i.e.. when the guide lines 21 or the 
represented by each main scanning line image data cannot synchronous marie airays 25 are slightly destroyed, the 
be clearly determined. However, when the positions of fee 40 apparatus of this embodiment can cope with tiiis and can 
two guide lines 21 on the main scanning line image data are provide a coaect decoding result as long as a read image is 
detected, a position (central position of main scanning data distorted to a degree as shown in FIG. 3. 
sanc^ling) on each main scanning line image data corre- Image data decoding xxrocessing according to this embodi- 
sponding to the vertical central position (or may be a vertical ment will be described below. 

mesh range) of meshes in the mesh pattern 22 can be 45 FIGS. 4A and 4B show a flow chart of processing for 
determined based on the detection results in association widi storing image data obtained by scanning the encoded image 
the positional relationship of an original image, i.e., fee 20, and recognizing fee guide lines 21 as main scanning 
predetermined positional relationship farmed between fee refercoce patterns from fee stored image data, feereby 
two guide lines 21 and fee mesh pattern 22 in the encoded main- scanning sampling image data. FIGS. 5 A and 5B show 
image on fee recording medium as an original. For cxait^e, 50 a flow chart of processing for recognizing fee synchronous 
in fee image format shown in FIG. 2. fee two guide lines 21 mark arrays 25 as fee sub-scanning reference patterns from 
are parallel to cacfa ofeer. and are also parallel to fee mesh fee main-scanning sampled image data, and sub-scanning 
pattern 22. Since a straight line connecting one point in fee san^}ling fee image data. This embodiment has a character- 
upper guide line 21 and one point in fee lower guide line 21 istic feature in recognition of fee guide lines 21 and fee 
passes a plurality of meshes arranged at equal intervals in fee 55 synchronous mark arrays 25. Even when fee guide lines 21 
mesh pattern 22, line image data expressing an image of or fee synchronous marie arrays 25 arc slightly destroyed, fee 
such a straight line shoild also have fee same feature. destroyed position can be determined wife high precision. 
Therefore, points of fee two guide lines 21 arc detected from In steps 4-1 to 4-5 in FIG. 4A, fee encoded image 20 on 
fee line image data, and an interval between fee two points fee recording medium is read by fee image sensor 11. and fee 
is equally divided to obtain fee main scanning central 6o read image data is written in fee image RAM 15. A scanning 
position of nteshes in fee line image. end condition (memory full) in step 4-3 is merely an 
The synchronous mark arrays 25 as fee sub-scanning example, and scanning end may be determined in response 
reference patterns are utilized to deiOTnlne fee positions of to various ofeer appropriate events. The image RAM 15 is 
meshes in fee sub-scanning direction in fee read Image data. assumed to comprise a byte memory as In step 4-4. FIG. 6 
More specifically, clocks of fee upper and lower synchro- 65 shows a memory map of fee image RAM 15. A one-line 
nous mark arrays 25 are detected from the image data, and image (line image) is written in one of 1 horizontal lines (1 
corresponding upper and lower clocks arc checked. In fee successive addresses of fee image RAM 15) in FIG. 6. 
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The image decoding operation starts from stq) 4-6 in FIG. d estroyed is checked In FIG. 8, die imy c d ata is divide d 
4A. In step 4-6. a guide line set (pixel group featuring the into_cigllt searching bl ocks^ 84 . 'it ie sTzTTdcptfa) of fi e 
guide line as the main scanning Fcference) is searched from scafQimg~b lbclHs determined in step 4-13, and die first 
the entire image data sU^cd in the format shown in FIG. 6. s earching biocfc startmg bo m the line next to the start line82 
As shown in FIG. 2, the guide line 21 is a black continuous 5 is ^sclccted. The depth ot the searc hing block can be 55g- 
line, and has a feature whidi is not present in other elements mined in consideration of (he guide Jinc set infomation 
of the encoded image 2^, Therefore, for example, the guide o btained in step 4-6, a i»occssinjE time. preasiOa'ancft he" 
line set can be defined by three, Le., white, black, and white like, and this v alue dcteimiacs a necessary length of the 
parallel run lengths 73, 74, and 75 having an appropriate w hiteTglack, and white run lengths associated with die gu ide 
interval, as shown in FIGS. 7(a)-7W. An initial value of an lO li ne set searcfted in step 4-i4j More speci fically, when run 
interval or width 76 of the run lengths necessary for finding l engths having an appropriate interva l and fiaving^le ngth 
the guide line set may employ a fixed standard limit value or equal to or larger tban me depth of the searching block arc 
may be determined by measuring the widths of blade or dc ^cd as the white, black. ancTwlutc run Icnettisln the 
white meshes which most frequently appear in a line image. depth direction, it can be dctcnnined thatthe guideHiesets 
The guide line set defined by the three ^ypropriate run is are present and data representing their positions (a rectan- 
lengths 73, 74, and 75 can be searched as follows. That is, gular area suirounded by the guide line sels) i$ stored. When 
an image is traced from the positions of white, black, and searching is unsuccessful, since it indicates that the guide 
white noeshes having an aj^opriate interval on an appro- lines <^ the searching block are damaged, a flag according to 
prlate line image on the image RAM 15 in a depth direction an unsuccessful state indicating for whidi one of the rig^t 
(vertical dh^ction in FIG. 6) perpendicular to the line Image 20 and left guide lines searching is unsuccessful or that search- 
to detect the numbers of successive white, black, and white ing is unsuccessful for both the guide lines is set so that 
meshes (run lengths), and the detection results are compared guide line positions can be interpolated later (4-15, 4-16). 
with a condition of the guide line set This processing is For example, in FIG. 8, since a contanunation SS is attached 
repeated. When seardxing is unsuocessfiil, an error occurs to the right guide line portions of the fourth and fifth 
since decoding is inqrassible (4-7). However, when search- 25 searching blocks from at>ove, the right guide line portions in 
ing is successful, the width of the guide line and the standard these searching blocks are detected as errors, 
value of an inclination of the scanning direction are deter- After the state of the guide lines is checked for one 
mined based on the informatioii of the guide Une set. In ste^ searching block, the next searching block is selected (4-17). 
4-8, margins taking a variation firom the standard value of and checking is repeated unt0 the last searching block at the 
the inclination of the scanning direction into consideration 30 end line 83 (4-18) is checked. 

are added to the ligjit and left (upper and lower in FIG. 2, but Although not executed la the flow shown in FIG. 4A, the 

'Wght and left^ will be used according to FIG. 6) guide line size of each searching block may be variably localized 

set positions, thereby determining right and left searching according to die searching result. For cxarrq>le, if a guide 

areas (searching widths 81 in FIG. 8) of image data in the line crrcr is detected in a certain searching block in step 

following processing. It is necessary to define the searching 3S 4-15, the depdi of the searching block is halved, so that 

widths 81 when images such as diaracters and the like guide line searching is executed for the two half-depth 

(which may cause noise during decoding) arc present around searching blodcs again or the loop including steps 4-13 to 

the encoded image 20, as shown in FIG. 8, However, tiiis 4-18 may be restarted using half-dcptii searching blocks, 

processing is not necessary when a blank space is formed From stqj 4-19 to the last step 4-31 in FIG. 4B, the ri^t 

around the encoded image 20. 40 and left guide line positions in each main scaiming line 

In steps 4-9 to 4-11, processing fw defining a searching image are determined for each searching block, and a main 

area of Image dau in a depth direction is executed. For this scaiming san^ling position of each line image (die entin^ 

purpose, the data start and end mariu 23 and 24 are detected path of the sampling positions is denoted by reference 

to obtain left and right guide line positions where these numeral 31 in FIG. 3) is obtained by an equal dividing 

marks 23 and 24 are detected. The data start and end marks 4S m^od. Image data bits (pixels or meshes) at the sampling 

23 and 24 can be drtected by detecting a line image positions are extracted from the line images to form main- 

induding periodic black and white patterns. For example, in scanning decoded orays 100 (FIG. 10). The guide line 

&e encoded image shown in FIG. 2, the following single positions (central positions in the guide line widths) on each 

conditioa for detecting these marks can be sufficiently used. main scanning line image are determined by actual mea- 

That is. since each of the start and end marks 23 and 24 50 surement when no failure flag indicating an error or desCruc- 

consists of 24 pairs of black and white rq)etitive patterns, tion is set for the guide line portions of a searching block to 

when about 20 pairs of black and white patterns having the whid) the main scanning line image belongs. However, 

same cyde arc found in consideration of a margin, it can be when the failure flag is set, the guide line positions are 

determined that these marks are detected. The data start determined by interpolation based on other guide line posi- 

mark 23 is preferably searched ftom an upper line (first line) 55 tions actually measured from normal guide line portions. For 

of the image data, and the data end mark 24 is preferably example, as shown in FIG. 4B. a list of failure flags are 

seardicd firom a lower Une (last line) of the image data. looked up. and guide line positions in a main scanning line 

When the start or end mark 23 or 24 cannot be detected, image of interest can be obtained by linear interpolation 

error processing is executed since it can be considered that fi:om Guide Une positions in main scanning Une images at 

the error is caused by an erroneous operation such as partial «o the ends of previous and foUowing searching blocks, 

scanning of die encoded image 20 (4-10 or 4-12). According to the flow of FIG. 4B. NO is determined in 

^ 4-18 Jmagcj(l ata in which the searching step 4-20 when both the right and left guide Une portions of 

wmm «1 arid the searching d^Uftom a start line 82 to an' a current searching block are correa (it can be determined 

end line i n FIG. 8) are defincjTby fe e abo ve-mentioned since the corresponding faUure flags are reset). In this case, 

pr ocessingk^^ bd i nte> a pl uraU ^crf part ial Imagcjeg- 65 the central positions of the right and left guide line portions 

i^^Jn qif^>th dif^on. whicn_a tc^^ed sc^^m fg of each main scanning Une image in the current searching 

b^^Tand wHRher or not the guide fine in each segment is block are actually measured in step 4-29. and an equally 
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divided position according to die encoded image format this time, a read error occurs. When the dock length falls 

between the two central positions is then obtained as a main within the range of the standard value, values of meshes on 

scanning sampling point Thereafter, an image bit at each a straight line connecting the central positions of the right 

sampling point is extracted. If it is determined in step 4-20 and left start clocks are extracted from the main-scanning 

that the failure flag of at least one of the right and left guide 5 decoded image data 190 to obtain data representing black 

line portions is set, the flow advances to step 4-21 and and white levels oi meshes of the first column in the mesh 

subsequent processing. In order to interpolate the guide line pattern 22 in order to perform sub-scanning saixq)liDg. In 

position for which the failure flag is set, the guide line addition, »he feature parameters (the central positions, the 

position imjnediatcly preceding the current searching block. lengths, the inclination, and the like) are set as the standard 

i.e., the guide line position in the last main scanning line lo values for the next clocks. 

image in the previous searching block (or the guide line In step 5-5, the length of the previous dock is added to the 

position of the start line obtained in step 4-9 if the current central position of the previous clock to predict the central 

searching block is the first searching block) is determined as position of the next dock. In this embodiment, this predic- 

an interpolation start point (4-21), A searching block includ- tion can be made since the synchronous mark arrays of the 

ing normal guide line portions is searched from the follow- 15 encoded image 20 (FIG. 2) comprise repetitive patterns of 

ing searching blocks (4-22, 4-26), and the guide line position black and white clocks having the same length, and hence, 

in the uppermost main scanning line image in the normal the clock length is equal to the clock interval. In step 5-6, 

searching block is detected as an interpolation candidate searching is vertically performed from this predicted point 

(4-27). The guide line positions on the main scanning line along the mesh array of die synchronous mark array on the 

imagesin the searching block of interest present between the 20 main-scanning decoded image data until a mesh value 

interpolation start and end points are determined by differentfromthatof the predicted point is detected, thereby 

interpolation, and main-scanning sampling is ihen executed actually measuring the next clock. For example, if the 

(4-28). Although not shown in the flow, when one guide line predicted point is **r representing a black pixel, vertically 

portion in a certain searching block is normal its position is successive "l^s having the predicted point as the center are 

directly measured. When a guide line portion near the end is determined as Ihe next docks. The length, the center, an 

line is destroyed, the guide line position evaluated in step indination between the right and left clocks, and the like of 

4-11 is used as an interpolation end point (4-23, 4-24). the next docks are obtained as actual nacasuiement resiilts 

A guide line section induding an error is interpolated on (5-7). When prediction and actual measurement arc 

the basis of the positions actually measured in other normal executed in this manner, if a clock includes no eircr, the 

guide line sections, thus executing hi^-precision main- 30 predicted central point and the actually measured central 

scanning sampling. As a result, the anays of die main- point should fall widiin a predetermined range. However, if 

scanning decoded image data 100 are coit^leted, as shown the dock is destroyed, its dock length and inclination must 

in FIG. 10. Columns on two sides of the main-scanning be considerably changed from the standard values (the 

decoded inoage data 100 are linear image mesh arrays length of the previous dock and the inclination between the 

extending along the main scanning center path 31 (FIG. 3) 35 previous right and left docks), as shown in FIGS, 9 and 10. 

of the synchronous mark arrays 25 as the sub-scanning In FIG. 9, since a contamination 63 paints a black clock 25a 

scanning reference patterns. In the flow chart of FIGS. 5 A of the right synchronous mark array 25, a black mesh array 

and 5B, the mesh airays of the rig^t and left syndironous indicating a piece of the contamination 63 surrounded by a 

mark arrays 25 are checked to detennine the sub-scaiming meshed line C in FIG. 10 is formed in the mesh array (pixel 

central positions of the clocks in Ihe synchronous mark 40 array on the scanning path 31 shown in FIG. 9) of the 

arrays 25, and the mesh pattern 22 is dien subjected to synchronous mark array 25 on the main-scarming decoded 

sub-scanning data sampling to identify black and white image data 100 shown in FIG. 10. Therefore, the length of 

levels of meshes. the next dock is observed to be longer by the contamination 

In step 5-1 in FIG. 5A. the length in the depth direction than that of Ac previous clock (white dock and its actual 

of a dock serving as a standard value (comparison reference 45 measurement center is indicated by PI in FIG. 10) in the 

value) of the start clock in stq) 5-3 is determined on the basis mesh array of the right synchronous mark array 25. A 

of the main scanning width, Le., an interval between the predicted center P2 of the next dock is largdy deviated from 

light and left guide lines 21 (obtained in FIGS. 4A and 4B). an actually measured center P3. Since a corresponding 

In this embodiment the image sensor 11 is assumed to be portion of the right syndironous mark array 25 has no 

manually scanned on the encoded image 20, and a dock 50 contamination, a difi'erence between a center P2 predicted 

length may be Considerably varied. Therefore, a rdatively from the previous clock (its center is indicated by PI) and an 

large margin must be added to ttie standard value. Note that actually naeasured value P3 of the next dock is very small 

in place of calculating the standard value from the main if there is any. Therefore, the feature parameters predicted 

scanning width, the mesh arrays of the syndironous mark for the next dock are conned with actually measured 

arrays 25 on the main-scanning decoded image data 100 are 55 feature parameters, and a difference therebetween is checked 

diecked to obtain an average black or white run lengths and to detect a dock error caused fary, e.g., contaminarion. In the 

die obtained run length may be used as the standard value. flow chart of FIG. 5B, a difference between the predicted 

In step 5-2, the first (black) docks of die right and left central point of the next dock and the adually measured 

syndironous mark arrays 25 on two sides are detected from central point of the next dock for each of the right and left 

the mainscanning decoded image data, and their central 60 synchronous marks is obtained (5-S), and it is checked if the 

points, lengths, an inclination between the right and left first difference falls within an allowable range (5-9, 5- 10. 5-13), 

docks with respect to the main-scanning direction of the thereby discriminating whether or not the next clock is 

image data 100 (horizontal dtrection in FIG, 10), and the like proper. Step 5-11 is executed when ndther the rigjit nor left 

are actually measured. In step 5-3. it is diecked if the dock clocks are proper. In this situation, since only die previous 

length of die measurement results falls within the range of 65 state of the synchronous mark array 25 is reliable, the rig^t 

the standard value obtained in step 5-1. When die dock and left predicted points obtained in step 5-5 are determined 

lengdi does not fall within the range of the standard value at as the central points of the next clodcs. In step 5-16, a mesh 
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array on a straight line connecting the central points is portion other than the destroyed portion is allowed by a 
sanq>led from the nudn-scanning decoded image data 100. segment mcttiod, and the guide line position of the destroyed 
and the sampled data array is determined as data indicating portion is interpolated based on the obtained position infor- 
blackand white levels of associated meshes. Steps 5-12 and mation. For the synduonous mark anays 25 as the sub- 
s' 14 are executed when one of the right and left next clocks 5 scanning reference patterns, the positions of the clocks are 
is proper, and the other is not proper. lo this case, the central obtained by prediction based on feature parameters oi an 
point of the improper clock may employ a predicted point adjacent cLodL actual measurement from the predicted 
from ttie previous clock. However, in order to improve point comparison t>etween the actual measurement result 
precision* the central point of the improper clock is obtained and the prediction result, and predetermination of a position 
from the acmaUy measured central point of the proper clock lO whose conqmison result indicates a clock error. TlKiefore, 
on the basis of the inclination. For exanq)le^ the inclination the positions of ikt sampling refmnce patterns can be 
of right and left pfx^r white clocks observed in ah vpper determined from image data with higher predsicm even in a 
portion of FIGS. 9 and 10 can be evaluated by a vertical relatively strict use environment, e.g., the presence of mis- 
diffexence of 2 (meshes) between the central points PI of leading characters, a variation in scanning direction, 
right and left clocks. Of black clodcs located immediately 15 contamination, and the like, and black and white levels of 
below these two white clocks, the right black dock is meshes of the mesh pattern can be identified with high 
inqnxiper, and (he left black clock is proper. The central pc^t precision on the basis of the detennination result, 
of the left black dock is acmally measured as P2 in FIG. 10. Second Embodiment 

Therefore, a position two n^eshes above an intersection FIG. 11 shows the overall arrangement of a data reading 
between a horizontal line passing through the actually 20 apparatus 110 according to the second embodiment of the 
measured point and a straight line as the mesh array of the present invendon. In this embodiment, an image sensor 111 
right syndironous mark array 25 (in this case, ttiis position mechanically scans a recording medium 118> As shown in 
fortunately coincides with the position P2 predicted from the FIG. 11, a controller 112 drives a stq)ping motor 117. The 
center PI of the right previous dock) is detennined as the ste^^g motor 117 moves the image sensor 111 via pulleys 
central point of the right improper dock. In step 5-15, only 25 117a, 117b and a bdt 117 c along a predetermined path, e.g., 
the dock length of die proper dock is updated as the a rail ^th this arrangement, since variations in moving 
standard value of a dock intervaL In step 5-16, a mesh array speed and direction of the image sensor 111 are considerably 
between the centers of the right and left clocks is sampled. reduced as ccHnpared to a manual operation, a distortion of 
When both the docks are proper, the actually measured read image data can be remarkaHy reduced (such a merit is 
dock centers are settled, and a mesh array therebetween is 30 attained even when die motor 117 is not controlled by a 
extracted. In this case, all the standard parameters are special sarvo mechanism or the like). Thaeforc, by utilizing 
iqxlatcd by feature parameters of the actually measured this nature, some of constituting elements and processing 
docks in step 5-15. operations described in the first embodiment can be facili- 
In step 5-17. end of decoding is checked. In this step, the tated or omitted. For example, even if an encoded image 120 
number data determined according to the format of the 35 having no data end mark shown in FIG. 12 is used, die 
encoded image is compared widi the number of execution number of lines bdow a data start mark 23 until data ends 
times of decoding step 5-16. When the number of execution can be roughly predicted based on a prcdsc scanning speed 
times of step 5- 16 is equal to the number of data determined of the image sensor 111. Even if the size of each searching 
by the format, and the mesh array of the following synchro- block is increased, the positicm of a destroyed guide line 
nous mark axray 25 does not include a clock up to the end 40 portion can be interpolated with high predsioa In recogni- 
line (5-18), i.e., when the number of docks determined by tion processing of synchronous mark arrays 25 as sub- 
actual measurement or interpolation from the mesh array of scaiming reference patterns, not local parameters, e.g., fea- 
the synchronous mark array 25 on ttic main-scanning ture parameters of an immediately preceding dock but 
decoded image data 100 is equal to the number defined by global parameters avexaged from the entire image can be 
the format it is determined that propa* processing opera- 4S used as comparison standard values. Thus, the position of 
dons aie executed, (bus con^letittg saiqptling processing. the next dock be predicted and evaluated based on the 
When erroneous clock interpoladon or the like is executed global parameters with high predsioo. 
during sampling processing, the end line of the image data Another Arrangement 

arrays 100 is readied tiefore the number of data defined by The first and second embodiments have t>een described 

the foimat is obtained (NO in step 5-19), or another clock is so above. Various dianges and modifications of the present 

found after die number of data defined by the format is invention may be made within die spirit and scope of the 

obtained (NO in step 5-18). Thus, an error can be detected. invendon. 

As described above, according to the first embodiment The data start and end marks 23 and 24 described in the 

sampling reference patterns (the guide lines 21 and the first embodiment arc effectively used to predict dieir coire- 

syndironous mark arrays 25) induded in the encoded image ss sponding pardons (executed in steps 4-9 and 4-11 In FIG. 

20 are improved in consideration of reading of the encoded 4A) when the corresponding portions a[ the marks are 

image 20 under a considerably strict envLronmental destroyed. However, the normal functions of the marks 23 

condidon. e.g., a diange in scaiming speed or direction and 24, Le.. the functions of indicating start and end of data 

during scanning which often occurs since the encoded image can be realized without using these marks. For example, if 

20 is manually scanned by the image sensor 11. characters 60 the data start and end marks 23 and 24 are omitted from FIG. 

and the like whidi cause noise and are recorded around die 2. successive white |»xels appear between black pieces of 

encoded image 20. and the like. In addition, a countcrmea- the two guide lines 21 in a main scanning line image 

sure against a case wherein the sampling reference patterns obtained by scanning these portions (white portions), thus 

are partially destroyed is taken in recognition of the sam- detecting start and end of data. 

pling reference patterns in the read unage data. That is, for 65 In the above embodiments, the two guide lines 21 are 

die guide lines 21 as the main scanning reference patterns. arranged outside die mesh pattern 22 as the main scanning 

measurement of the positions of the guide Unes 21 in a reference patterns in the encoded image 20 or 120. However. 
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various other appropriate main scanoiog reference patterns 
may be used. For exanqile, as shown in FIG. 13, when three 
(or more) parallel guide lines FR are used as main scanning 
reference patterns for a mesh pattero MP. when the guide 
lines PR is partiaUy destroyed, the guide Line position can be 
more strongly restored. For exanqile, when one of the three 
guide lines PR is partially destroyed^ the destroyed guide 
line portion can be accurately determined based on the 
corresponding positions of the remaining two guide lines. As 
exentplitied by the central guide tine PR in FIG. 13. a main 
scanning reference pattern may be arranged in the mesh 
pattern 22. When the scanning direction of the image sensor 
is stable, the main scanning reference pattern may be con- 
stituted by one guide line FR extending along the main 
scanning direction of the mesh pattern NfP. as shown in FIG. 
14. In this case, the scanning direction (moving direction) of 
the image sensor can be determined by checking a main 
scanning line image passing through the guide line PR. By 
checking the main scanning line image, the end position of 
the guide line PR can be determined. Thus, the central 
position of eadi mesh in the main scanning direction on the 
read image data can be detemiined on the basis of the above 
dctcimination result and the positional relationship between 
the end points of the guide line PR d^oTnined as an image 
format and meshes of the mesh pattern MP, thus executing 
main scanning sampling. Furthermore, it is advantageous 
that the main scanning reference pattern comprises a con- 
tinuous line in black or in a color different from surrounding 
colors^ but may comprise a discontinuous mark. 

In the encoded image 20 or 120 of the above 
embodiments^ the two synduronous mark arrays 25 in which 
Mack and white meshes (clocks) alternately appear on the 
two sides of the mesh pattern 22 are used as ttie sub- 
scanning reference patterns. However, various other appro- 
priate patterns syndu-onous with sub-scanning mesh arrays 
(which can have a positional correspondence) may be used 
as sub-scanning references. Fc^ example, when the scanning 
direction of the image sensor is sU^le, one synchronous 
mtark array SR extending along the sub-scanning direction of 
Ihe mesh pattern MP can be sufficiently used, as shown in 
FIG. 15. In order to restore partially destroyed clocks with 
higlker precision, three or more synchronous mark airays 
may be arranged to be separated at given intervals in the 
main scanning direction. The length of each clock in a 
synchronous mark can be arbitrarily determined. For 
example, the sub-scanning reference patterns may be 
formed, as shown in FIG. 16. That is, synchronous marks SR 
in each of which clocks fanning an elongated black piece are 
aligned may be arranged above and below the mesh pattern 
MP to have the rclationsh^ that a line connecting corre- 
sponding clocks passes through a boundary between adja- 
cent meshes of the mesh pattern MP. Alternatively, as shown 
in FIG. 17. in addition to a first clock mark array SRI. a 
second dock mark array SR2 in which each clock mark is 
arranged every predetermined number (8, in this case) of 
dock marks of the first clock mark array SRI may be 
provided. With Ihese clock airays, a sub-scanning reference 
for the mesh pattern MP may be formed. In ^s case, the 
number of docks can be easily managed in an image 
decoding operation* and partially omitted dock marks can 
be more reliably restored. Fes example, a clock mark 
counter for the first clock mark array SRI is initialized at a 
position where a coinddence between docks of the first and 
second dock mark arrays SRI and SR2 is found on the read 
image data. Thereafter, every time a dock mark on the first 
dock mark array SRI is detected or is determined by 
restoration according to this embodiment, the dock mark 
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counter is incremented When a coinddence between clocks 
of the first and second dock mark arrays SRI and SR2 is 
detected again, the dock mark counter is checked. In the 
format shown in FIG. 17, if the content of the counter is 

5 equal to 8n (n is a natural number), it can be determined that 
the first clock mark array is ^propriatdy recognized. If a is 
2 or more, it can be evaluated that the second clock mark 
array SR2 is partially omitted. If the content of the counter 
is not equal to Sn. it can be determined that the tint dock 

10 marks arc erroneously recognized, and an eiror can be 
detected As a result, when the fint clock mark array SRI is 
partially destroyed and clocks arc restored through predic- 
tion of the next clock position, actual measurement, 
comparison, and reevaluation processing operations (see 

15 FIGS. 5 A and SB) described in the above embodiments, an 
error occurring in the restoration process can be restricted to 
a narrow local area as a dock interval of the second dock 
mark array SR2. Furthermore, clock positions of the 
restricted portion can be obtained by equally dividing an 

20 interval between the two clocks of the second clock mark 
array SR2. Therefore, black and white levels of meshes of 
the entire mesh pattern MP can be identified while recog- 
nizing such a local error (in the above embodiment, an error 
is detected in the whole system as shown in steps 5-17 to 

25 5-19 in FIG. 5B, and decoding is disabled). In a practical 
application, a suffident correction opportunity can be given 
to black-and-white identification data at a position where die 
above-mentioned local error Is found in error correction 
processing using these checking codes. 

30 The above embodiments are made under an assumption 
that images sudi as characters whidi are not suitable for 
decoding are present around an encoded image of course, 
sudi an assumption need not be made. In this case, recog- 
nition of san:4>ling reference patterns can be considerably 

35 facilitated, and conditions of reference patterns (conditions 
of aaangements. geometrical shapes, and the like) for dis- 
tinguishing reference patt^s from images such as charac- 
ters can be rendered less strict. 
In the above embodiments, the main scanning sampling 

40 reference patterns (guide lines 21) and the sub-scanning 
sampling reference patterns (synchronous mark airays 25) 
are realized by s^arate constituting elements on an image. 
However, a san^iling reference pattern which can be com- 
monly used as these patterns may be used. 

45 Referring back to FIG. 16. for exami>lc. the synchronous 
mark array SR may be used as a conmion main scanning/ 
sub-scaiming reference mark. In FIG. 16, the synchronous 
mark array SR is present outside the mesh pattern MP as a 
data body. Therefore, if there is no unnecessary image 

50 exduding the mesh pattern MP and the syndironous miark 
array SR. it is not difficult to detect the synchronous mark 
array SR from the read image data. For example, paying 
attention to each main scanning line inoage of the read image 
data, black pixels appearing at two ends of each main 

55 scanning line image suggest a possibility of the presence of 
pieces of the synchronous mark array SR. In manual 
scanning, each main scanning line image is normally read 
not in a perfectly vertical direction but in a slightly inclined 
direction with some variations. Therefore, if several (two or 

60 three) main scanning line images having black pixd groups 
near their two end portions, successively appear, they Indi- 
cate a high possibility of the dock marks of the synchronous 
mark array SR Of these black pixd groups, black pixels 
groups which are distributed on image data to have a certain 

65 period can be surely determined as clock marks. Thus, 
paying attention to black pixel groups for several lines, if 
their centers are actually measured the measured centers can 
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be detemuDed as ttiose of clock maiks. A destroyed dock bility. Far example, when an error is detected when black- 
mark can be detected as follows as in the corresponding and-white identification data of a certain block is checked 
processing in FIGS. 5A and 5B. That is. the position of the according to a checking word, if the block includes a 
destroyed dock mark is predicted from an immediately position where a contradictioa is found, this position is 
preceding dock mark and its interval or pitch (a distance 5 predicted as a position where the error occurs, thus allowing 
between the centers of the immediately preceding dock so-called erasure correction (error a»rection when an error 
mark and a clock mark preceding the inunediately preceding position is known). 

dock mark), and is actually measured using the predicted If a sanyling reference pattern and a mesh pattern as a 

position as the center. The predicted value is conqiared with data bodty has a de&nable positional relationship, the central 

the actual measurement value to detect if a difference lo position of each mesh on the mesh pattern on the image data 

ttierebetween Is large. If the difference is large, the predicted can be obtained from the defined positional relationship and 

point is determined as the center of the current dock mark, position data of the sampling reference pattern obtained on 

or a position determined acccM-ding to the center of the center the image data. This will be described below with reference 

of the current clock mark in another array on the basis of an to FIGS. 20 to 22. In this case, a description will be made 

inclination between preceding clock marks is determined as 15 under an assumption that a scanning speed and a scanning 

the center of the current clock mark. When the central direction of the image sensor with respect to an encoded 

position of each clock mark is determined in this nmner, image indude certain variations. For the sake of sin^lidty. 

corresponding dock marks are selected fr^om the upper and two synchronous matk arrays induding clock marks and 

lower synchronous mark arrays SR. and their centers are separated at a given interval are used as sampling reference 

connected by a straight line. T he straig> ^t Kitf* k ^jjiifllly 20 patterns. A mesh pattern is arranged to be sandwiched 

d ivided according to the number of image meshes, a nd between these two synchronous mark arrays, 

image bit s present a t equally" divided positions are read to FIG. 20 shows the positional relationship on image data 

i d^ntv DJack and white levels at the centers of meshes (th e read from the image sensor. Reference symbols U, and U^^.i 

last desc ription assumes a case wherein each dock d the denote the central positions of ith and (i+l)th clocks of tfie 

sy ncnropous mark. array SR in FIG. 16 i s shifted b y half a 2s upper synchronous mark array; and and , the central 

mesh in the ho rSontal direction to coincide with the cent er positions of jth and (}4'l)th docks of the lower synchronous 

of eac h naesh ot tfaf vn^^ pattern x^p) mark array. For example, the central positions and Bj. and 

As shown in FIG. 16, when the longitudinal direction of 11^.^^ and can be those of adjacent clocks which have a 

e ac& dock Coincidg s wititi ^^'^ ^"^4J^ p tCwcai adjac ent minimum sub- scanning distance on the read image data. In 

m^hes of the mesh pattern S n ^e^ i i^. i . R ^. ^ qj^. 1 ^ FIG. 20, the coordinates of the point U; arc represented by 

o f ith and {j+lVth clock marks arc selected from the upper an origin (0.0), and the coordinates of the point U^^ are 

and lower syndironous mark arrays SR. and each central represented by a point (xl.O) on the X-axis. The positions on 

position of a target vertical mesh amy can be calculated the image memory are mcrdy normalized and expressed for 

from these position data. This will be described below with the sake of simpUdty. The Y-axis direction corresponds to a 

r eference to FIGS. 18 and 19^ In FIG. 18, the centers Uj and S5 direction of a main scamiing line image, 

-^-^t^i (positions on the read image data) of the ith and (i+l)th FIG. 21 shows the positional relationship on a recording 

dock noarks in the upper synchronous mark array SR are medium in cocrcspondcnoe with FIG. 20. Reference symbols 

coimected. and their middle point CI is calculated. T^ and T^i denote the centers of idi and (i+l)th doclis in die 

Similarly, the centers B and B,^i of the ith and (i+ I)th dock upper synchronous mark array; Ay and A^i . the centers of jth 

marks in the lower synchronous mark array SR are con- 40 and (j+l)th dock marks in the lower synchronous mark 

nected to calculate their middle point C2. FIG. 19 shows a array. The dock centers {TJ and {A,} of the synchronous 

portion including the noiddle points CI and C2 of the read mark arrays on the recording naedium, and a central position 

image data. Each horizontal line in FIG. 19 corresponds to {P} of each mesh of the reference pattern are already known 

one main scanning line image. One cell in FIG. 19 indicates items for the data reading apparams, and the apparatus has 

a 1-bit ( 1-pixd) memory cell. The middle points CI and C2 4S these data as an encoded image model For exan4>le, data of 

are connected by a straight line, and the line is equally all diese positions may be stored, or when intervals between 

divided according to an image format thereby determining positions have a predetermined synchronous rdationship. all 

the central position of meshes in a target mesh array on the the central positions can be calculated from one or a plurality 

image data memory, i.e., a storage position (indicated by a of position data and synchronous data. In FIG. 21, a line 

mark "x" in FIG. 19). With the above-mentioDed middle so connecting the centers T, and T^^ on the upper synchronous 

point method, since a data sampling point is obtained from mark array and a line coimectlng the centers Ay and A^, on 

two pairs of two adjacent dock centers, errors included in the lower synchronous marie array are illustrated to l>e 

positions U^, U^^^. B^, and B^^^ as synchronous mark array paralld to each other, but need not always b»e parallel to each 

SR measurement points (or intcqxilation points) can be other. In FIG. 21, the center of the ith clock of the upper 

absorbed and decreased by averaging. 5S synchronous mark array is represented by the origin (0,0), 

Points indicated by the marks **x** in FIG. 19 need only be .and the center of the (i+l)th clock is represented by a point 

data-sampled to identify black and white levels of meshes. (D.O) on the X-axis. 

Surrounding pixel values (e.g.. upper, lower, right, and left Assume that dock centers of the synchronous mark arrays 

pixel values) of the point indicated by the mark "x** are as the reference patterns are obtained from the read image 

checked, and black and white levels of meshes may t>e 60 data by the method as described above, and a rectangular 

determined based on the checking results, if so desired. image block surrounded by the four dock centers Uj. U^j. 

Alternatively, when surrounding pixel values contradict with By, and By^.^ (FIG. 20) is considered. The problem is how to 

^ the pixel value of the point indicated by the mark "'x'* detect die central positions of meshes present in this rect- 

obtained based on die recognition result of the synchronous angular image block. Thus, the data reading apparatus 

mark array SR. dato indicating tliis is stored, and can be 65 selects corresponding clock Centers T^.T^^j. A . andAy^i on 

effectively used in the error correction processing later using the encoded image model (i.e.. on tfie recording medium), 

the checking codes, thus enhandng coror correction capa- Since the encoded image modd iodudes position data {P} 
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of ihe centers of meshes in the entire encoded image, a set funher details about XI. X2, X3, Yl, and Y2). In this 
of central positions of meshes present in a rectangle (in this manner, ttic central coordinates (X.Y) of a corresponding 
case, a parallelogram) defined by T,, T^^. Ai. and A^^ can mesh in the target image block can be obtained from the 
be formed For cxan^lc, in the case of FIG. 21. a set of cocffdinates (x,y) of a mesh center in the model. Therefore, 
coordinates (x,y) which satisfy the following relatioD can be 3 an image bit present at the position (X,Y) is sampled to 
selected from a set {P} of central positions of all the meshes: obtain data indicating a black or white level of the racsh. 

Note that since an image frame farmed by the points T,. | , 
0£*fL)f5/) . actually an elongated small local area (see 

FIG. 2), an oflFsct AH is nomially considered to have a 
(L is the inclination) negligible magnitude ( AH=0> even when manual scanning is 

IntCTnal points of the rectangle defined by T(,Ti+i,A^ and 10 ^j^^j^^^^ Therefore, equation (2) can be simplified as 
A^i correspond to those in a rectangle defined by U,, Ui+i* Y=Yd. 

By, and B^^ on the read image data, and never fall outside described above, according to one arrangement of the 

it When the central positions of meshes present inside the present invention, image data obtained by reading an 
rectangle are detected on the x-y coordinate system of the encoded image on a recording medium is segmented into a 
encoded image model in this manner, the detected positions 15 plurality of partial image data blocks, and a feature pixel 
are coca:dinate-converted to positions on the X-Y coordinate group featuring a piece of a main scanning reference pattern 
system of the read image data, and image bits at the is searched using the segmented partial image data blocks as 
converted position can be extracted. searching units. A position of the reference scanning pattern 

FIG. 22(a>-22(0 show exan^les of coordinate conver- in a partial image data block for which searching is unsuc- 
sion. FIG. 22(a) shows an image block of interest on the 20 cessful is determined based on position data ctf normal 
model, and the center of a certain mesh is represented by pieces, indicated by the feature pixel groups for which 
coordinates (x,y). FIG. 22(/) shows a corresponding image searching is successful, of the main scanning reference 
biock (target image block) on the read image data, and pattern. Therefore, even when the main scanning reference 
coordinates cwresponding to the coordiDates (x,y) are rep- pattern is partially destroyed by, e.g., contamination on the 
resented by (X,Y). An operation executed between FIGS. 25 recording medium, its position can be obtained with high 
22(a) and 12(b) is hcffizontal scaling for matching the lengdi precision, and black and white levels of meshes of the mesh 
of the upper side of a rectangle, an operation executed pattern can be reliably identified on the basis of the position 
between FIGS. 22(5) and 22(c) is convcrsioD for matching data of the main scanning reference pattern, 
the inclination with that of the lower side (from Ui to By) of Furthermore,.according to one aspect of the present 
the target image block, and an operation between FIGS. 30 invention, when synchronous marks used for positioning a 
22(c) and 22(d) is scaling for matching the length of the left mesh pattern as elements of a scanning reference pattern are 
side with that of the left side of the target image biock With scardied from image data obtained by reading an encoded 
these operations, of the four apexes U^. U^^. By, and By^i of image including the mesh pattern and tfie scanning reference 
the target image block, three points U,, U.vi. and By coincide pattern, a position a synchronous mark to be determined 
witiicorrespondingpointsoftbcimagebiockaf interest As 35 next is predicted according to feature parameters of a 
shown in FIG. 22(e), a corresponding point of the image synchronous mark determined by actual measurement, and 
block shown in FIG. 22(d) need only be shifted to the point the synchronous mark is actually measured to have tiie 
B - 1 diagonal to the point U^, thus obtaining the target image predicted position as the center. The actual measurement 
bfock. An image block is a local area, and changes in speed result and the predicted content are conq)ared with each 
or direction in the local area are very smaU even in manual 40 other. When the two results essentiaUy coincide with each 
scanning. Thcccforc, convcnion from FIG. 22(<0 to FIG, other, the actually measured position is determined as the 
22(e) can be considered as a uniform distortion (distortion position of the synchronous mark. However, when a non- 
proportional to an area ratio) without posing any problem- coincidence is found, a mark destroyed portion caused by. 
As a result the target coordinates (X,Y) arc given by: e.g., contamination is detected, and the position of the 

45 synchronous mark is determined based on the already detcr- 

/ Xd+M Yd \ { Yd \ mined feature parameters. Therefore, even when the scan- 

x=Xd + iJ>y j • y -pj- j uijjg reference pattern is partially destroyed by contamina- 

tion or stain, the position of the corresponding synchronous 
Y=^Yd+AH'( ^^^^^7r^)'(-^-] mark can be determined with high jttedsion without omis- 

\ / \ J 50 sion. Thus, Wack and white levels of meshes of tiie mesh 

for pattern can be reliably identified based on the determined 

position data. 

SD = {X3-X2yxi (3) Third Embodiment 

^ = - *^ The third embodiment of the present invention will be 

. V (5) 55 described below. <Data Recording Apparattis> 

Xd = l J X + (M - L) y PIQ. 23 shows the overall arrangement of a data recording 

apparatus 1000 according to the third embodiment. The 
object of the entire apparatus is to record an image obtained 
by encoding data on a recording medium such as paper or a 
MuitipUcators for AD and AH, given by the above equations 60 sheet (which image is the same as that shown in FIG. 2 
(1) and (2) represent an area ratio of a rectangle in RG. described in the first embodiment and will be described 
22(d) indicated by a solid line, and a rectangle indicated by below with reference to FIG, 2). The characteristic feature of 
a dashed line and having coordinates (Xd,Yd) as a diagonal the data recording apparatus will be described in detail later, 
point AD is tiie difference between the lengths of the upper In FIG. 23, a CPU 1010 is operated according to a 
and lower sides of the target image block, and AH is the 65 program stored in a ROM 1020. In a data loading mode, the 
difference between the lengths along the Y-axis of the right CPU 1010 loads target data (e.g., a user program list) 
and left sides of the target image block (see FIG. 20 for suj^lied from an appropriate external device 1050 (e.g., a 
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persoDfll computer) in a RAM 1030 through a oommunica- 
tion iotcxface 1040. In a reoordiiig mode, the CPU 1010 
pcrfoims various error countenueasures of the target data 
stored in the RAM 1030 to form a data array which has a 
one-to-one coirespondence with image elements (meshes in 
this case) of an encoded image. Thereafter, the CPU 1010 
controls a printer 1060 to print the encoded image on a 
recording medium. 

The operation of the data recording apparatus lOOO, 
especially error countermeasures. will be described in detail 
below with reference to FIGS. 24 to 33. 
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In step 20-1 in FIG. 24, target data is loaded from the 
external device 1050 to die RAM 1030 through the corn- 
mumcation interface 1040. The bit matrix of this target data 
is denoted by reference numeral 300 in FIG. 25. When the 
encoded image 20 shown in FIG. 2 described in the first 
embodiment is assumed to be used, the fcnmat of the mesh 
pattern 22 as its data body coaesponds to a 4%>02 two- 
dimensional mesh matrix. Each mesh is an image element 
expressing one bit as a minimum unit of data. For example, 
a black mesh expresses a bit and a white mesh expresses 
a bit "0". When the 48x72 mesh pattern 22 is used, the size 
of the target data is given by 8 (bits)>^56 (256 bytes), and 
the data recording apparatus 1000 processes 2.048-bit target 
data do to d^o*? as the 64 (bits)x32 (bits) two-dimensional bit 30 
matrix 300, as shown in FIG. 25. 
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In step 20-2 (FIG. 24), $ bits in the bit matrix 300 are 
considered as 1 symbol, and target data So to S255 consisting 
of 256 symbols arc considered as an 8x32 two-dimensional 
symbol matrix 400 shown in FIG. 26. The size of each 
symbol is significant as a data unit (word) in steps 20-3 and 
20-4 (to be described later). Note that the present invention 
is not limited to 1 symbol=8 bits in step 20-2. 

The following relationsh^) is established between the 
two-dimensional symbol matrix 400 and the two- 
dimensional bit matrix 300: 
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where a as a primitive element of a Galois body GF (2*) is 
one of roots of a polynomial given by: 

a takes a value of 2 in decimal notation, and is expressed by 
a binary number as follows: 

000000 10(^ 

45 

In this connection, a*=a^4a^4<xVl= 16+844+1=29, and 
s„'=^j-^^^U^+^d^s+'^dt^+2!^dt^^l^dt^2^:^d^y+2'^ to a^^' take conesponding jaropcr values from 1 to 255 

(a"*=l). ^^O'^s of the right-hand sides of equations (7) and 
(8) indicate that a so-called syndrome is zero. 
In step 20-3 (FIG. 24), error checking codes arc added to 50 p^^^ exaii^)le, when equation (7) is solved fcr a, we have: 
the two-dimensional symbol matrix 400 of the target data to 

form a symbol matrix 500 with error checking codes shown IGM-AirM^iHWO (9) 

in FIG. 27. Arbitrary codes may be used as error checking 
codes. In FIG. 27. four checking symbols p are added to each 
flow (line) of the symbol matrix 400 of the target data, and 
four (decking symbols g are added to each of 12 columns of 
a two-dimensional matrix formed by the symbols i^. In ^ 
case, since four checking symbols (block checking 
codes) arc added per block (one flow or column), the four 
cheddng codes themselves have a double symbol eirar 
correction function. Furthcnnore. since two systems of 
checking codes g (flows) and ^ (columns) are added to the 
symbols £, quadruple oasure correction in each block of the 
target data can be realized. In the matrix shown in FIG. 27, 
checking symbols p and si are added assuming so-called 
Reed-Solomon codes. Its relation is as follows: 
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-continued 



otXB oi»» 01^ a2» 
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Equation (8) is similarly solved to calculate 

As a result of adding the error checking codes p AQd j^, the 
two-dimensional matrix 500 having a size of 12x36 symbols 
can be obtained. 
The above-mentioned block type oror checking codes 
&nd ^ are mainly used as countermeasures against random 
OTors, and are not effective for burst orors. For example, 
when a contamination becomes attached to a considerably 
wide contiguous range on an encoded image, an eiror 
beyond correctioD capability of the oror checking codes 
may occur. 

FIG 



30 



In step lo-4 in FIG. 24, countamcasurc processing 
against burst errors is executed to change the order of 
elements of the matrix 500 so t |jat a contiguous area on th e 
two-dimensional symbol matrix 500 obtained in step 20-5 
(pr ocessinjg for adding err^^ iec fc ing codcs)'B 'c onv^ d 
ingTaignrJ'^ «"^'*l' ^^^^ ^ <hr. enrodcd imag eT Prooessing 35 
in'step 20^ wilLbejcfenxa UQ as scrambling ot interleavin g 
K5iinaftcrrF IG S. 28A. 29A, ang^BUA respectively show 
RYm^I^mfl ^ces 600. 700. and SOO after first, second, and 
third scrambling op ecatHg ns^ Symbol matrices 600S, 700S, 
and 8005 'in FIGS. 28B, 29B, and 30B are simplified ones 40 
of the structures of the symbol matrices 600, 700, and 800 
to allow easy understandin g. In the first scramblin g 
op eration. Line data of the symbol naatrix 500 after the e rror 
c faecl^ftff codes are added is replaced That jy, ninth and 
tenth line data of the original matrix are moved to the third 4S 
and fourth lines of the original matrix 500, respectively; ^e 
third to sixth line data of the original matnx 500 are moved 
downward by two lines; eleventh and twelfth line data of the 
odginai matrix 500 are moved to the ninth and tenth lines of 
the originaL matrix 500, respectively; and seventh and ninth 50 
line data of the original matrix 500 are moved to the eleventh 
and twelfth lines, respectively, thereby foffraing a first 
scrambled matrix 600. Please refer to the matrix 600S in 
FIG. 2SB as a rough structure of the matrix 600 shown in 
FIG. 28A. In the second scrambling operation after the first 
scrambling c^eration, the lower half of the first scrambled 
matrix 600 is attached to the right-hand side of the upper 
half, thus forming a 6x72 (symbols) second saambled 
matrix 700. Finally, in the third scrambling operation, the 
second scrambled matrix 700 is moved in the horizontal 
direction by 16 symbols. As a result, a third scrambled 
matrix 800 shown in FIG. 30A is obtained. 

q[5eJasLsctamhlcd-m^^ 

r elationship as th e two-dimension^^^mgghjiatnx'onjh e 
_eQc^^^^^6r^d contiguous clcmcntson the scrambled 65 
matrix 800 SfeHmilarly contiguous elements on the encoded 
image. However, the matrix 800 has the relationship with the 
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matrix 500 before scrambling so that a contiguous pcNtion 
on the matrix 500 is located at discrete positions on the 
scrambled matrix 800. For example, for q^js* qi29' <li3o 
q^ji in the ninth to twelfth lines of the 33rd column on the 
5 matrix 500 before scrambling, q^zs (I129 located at 
the third and fourth lines of the 17th column, and q^j^ and 
q^ji are located at the third and fourth lines of the 53rd. 
column on the scrambled matrix 800. As a result, since a 
burst error on the encoded image is scattered as short eiiors 
OD the matrix 500 before scrambling, dror correction within 
the capability of the error (decking codes and g can be 
realized. As sdiematlcaUy shown by the matrix 800S in FIG. 
50B, a a portion occupies the central pcartion of the 
scrambled matrix 800 (FIG. 30A). This arrangement is made 
in consideration of a method of adding the error checking 
15 codes (FIG. 27) and a mesh data sampling method of a data 
reproducing ^aratus. More specificaily. as can be seen 
from HG. 27, two systems of error checking codes 5 and 
£ are associated with target data while only codes for 
checking horizontal bloc^ (lines) are added to the eiror 
20 checking codes fl. For this reason, even if a code crrar occurs 
in tills portion, there is no means for confirming the position 
of the error. As long as block checking cxkIcs or finite 
convolution checking codes arc used on a finite matrix, this 
problem is not perfectly avoided. In other words, on the 
25 matrix 500 shown in FIG. 27, an error at a symbol Sq 
reflected on both a syndrome value of the first line and a 
syndrome value of the first column (symbol values are not 
equal to zero), while an error at a symbol q^jg is reflected on 
only a syndrome value of a column (33rd columji) to which 
qi28 belongs. Fcm: this reason, when errors occur at three 
positions, e.g., the symbols qi^g, q,29. and q^o- correction 
is impossible. An error in an area of a is serious, and hence» 
its frequency is preferably decreased to be lower than that in 
other portions. Meanwhile, in main scanning decoding pro- 
cessing described later in a data reproducing apparatus, 
image bits at points obtained by equally dividing an imerval 
between the guide lines 21 are sampled using the upper and 
lower guide lines 21 of the mesh pattern 22 (FIG. 2) as the 
main scanning reference patterns. In this saji:4)ling method, 
an o£fset of an equally divided point from a true pasition is 
maximized near each guide line 21, and the central portion 
of the mesh pattern 22 has niaxinuun safety. Therefore, as 
shown in FIG. 30A, the error checking codes a arc arranged 
at the central portion of the matrix 800, thus decreasing an 
error rate of these codes a to be lower than those of other 
codes. 

The scrambled tw o-dimensioaal sw bol m^iix^ 800 is 
conv^ed" to a 48><72~tKfnwCRfifficSsioDal bit matrix 900 
while considering each symbol as vertical 8 bits, as shown 
in FIG. 31 (step 20-5 in FIG. 24). In this case, for example, 
Si28=|bob|b2b3b4b5b<jb7]^ (where T is the transposed 
mafarix). 
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Step 20-6 is executed to prevent ail aittt-wuigdjjjj^ 
variation of conversion characteristics of the inuige sensor 
used in a data reproducing apparatus. More specifically, 
photoelectric conversion characteristics of an image sensor 
cannot perfectly follow an instantaneous value of incident 
light Thus, when an image is scamied using such an image 
sensor, a conversion ou^ut varies between a case wherein 
scanning enters a white (or black) area after a long black (or 
white) area, and a case wherein a white (or black) area is 
kept scanned. For this reason, a white portion isolated in a 
black area is affected by the suirounding black area, and is 
detected while being reduced, or may sometimes be detected 
as a black pixel. As a result, accurate image data cannot 
often be provided. As shown in FIG. 2, the encoded image 
of the present invention comprises a two-dimensional black- 
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and-whitc mesh malrix of the mesh pattern 22. and each 
black or white mesh expresses one bit Therefore* it is 
preferable that pixel values arc accurately obtained* and this 
is necessary when each mesh is small and hcnoe, a recording 
density is high. Meanwhile, since data often have the same 
bit value, the above-mentioned d^ccdon error cf a pixel 
value tends to occur in such an area. In stq) 20-6 in FIG. 24. 
bit values oo the scrambled two-dimensional bit matrix 900 
are randomized using pseudo random numbers, so that the 
same bit value does not contlaue over a long distance, 
ttiereby solving the above problem. 

FIG. 12 shows an exao:^>ie of randcnnization. A pseudo 
random number generator P-RND EXORs an output from a 
D flip-flop D 15 of a 16-bit shift register (constituted by D 
flip-flops 1 to 16) and an ou^ut from a D flip-flop Dl. shifts 
bits of ttie shift register to the right, and ii^ts the EXORcd 
result to tiie D flip-flop Dl. A pseudo random number output 
md(n) is extracted from the D flip-flop D16. and is EXORed 
with a ccnesponding element bn on the bit matrix 900 ( 
Sn=bn 0rnd(n)). thereby randomizing the elements of the 
matrix 900. At the beginning of the operation, an appropriate 
initial value (e.g.. BBS (hex)) is preferably set in the shift 
register. As a result a 48x72 randomized two-dimensional 
hit matrix 1100 is formed, as shown in HG. 33. 

Tlie randomized two-dimensional bit matrix 1100 has a 
one-bit to one-mesh correspondence with the two- 
dimensional mesh matrix of the mesh pattern 22 to be 
recorded on the recording medium while positions on the 
matrices coincide with each other, and black and white 
levels of the meshes are specified by bit values. 

In order to achieve this, in step 20-7, the mesh pattern 22 
according to the bit matrix 1100 is printed on the recording 
medium. In step 20-7, sampling reference marks having a 
predetermined positional relationship with the mesh pattern 
22 are also printed. The sampling reference marks adopt the 
guide lines 21. the synchronous mark arrays 22, and the data 
start and end marks 23 and 24 shown in FIG. 2. As will be 
described later, a data rqiroducing apparatus finds these 
saizipling reference marks from read data of the encoded 
image 20. detcrmittes mesh positions in the mesh pattern 22 
with reference to positions of these reference marks, and 
samples image bits there. <Data Reproducing Apparatus> 

A method and apparatus for reading the encoded image 20 
(FIG. 2) recorded on a recording medium described above 
have afaready been described in association widi &c first 
embodiment and a repetitive descr^tion will be avoided. 

This embodiment will be described below with reference 
to the circuit arrangement shown in FIG. 1 A CPU 13 
operates according to a program stored in a ROM 14 (see 
FIG. 34). While an image sensor 11 scans an image, every 
time a controller 12 converts serial image data into parallel 
data, the CPU 13 writes the parallel data in an image RAM 
15 (14-1). Upon completion of image scanning, the CPU 13 
starts a decoding operation (14-2 to 14-6) of image data 
stored in the image RAM 15, and stores the decoded result 
in a RAM 16. 

The decoding operation for reproducing data is basically 
achieved by following encoding processing executed by the 
data recording apparatus 1000 in an opposite direction. First 
main scanning sampling step 14-2 (FIG. 34) and sub- 
scanning san^Hng step 14-3 are executed to sample image 
bits representing black and white levels of meshes located at 
the central positions of meshes from read image data, 
thereby obtaining a matrix corresponding to the last two- 
dimensional bit matrix 1100 described in data recording. 
I^ocessing operations in steps 14-1 to 14-3 are the same as 
those described in association with the first embodiment In 
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DC (dcrandomization) step 14-4. die elements on the matrix 
are derandomized using the pseudo random numbers 
described with reference to FIG. 32, thereby forming a 
matrix coiresponding to the non-randomized bit matrix 900 
5 shown in FIG. 31. The reason why the non-randomized bit 
matrix 900 is restored is that the randomization is given by; 

dcrandomization is given by: 

and. die two equations are derived fcr the same pseudo 
random numbCT md (n). The DC processing itsdf is the 

15 Same as randomization processing step 2<M», except that 
data to be HXORed with md (n) is b„ (non-randomized data) 
or S„ (randomized data), and a further descripdon thereof 
will be omitted 

After die non-randomized symbol matrix is obtained, 

20 processing (descrambling) Opposite to scrambling process- 
ing step 20-4 described with reference to FIGS. 28A and 
28B. HGS. 29A and 29B, and FIGS. 30A and 30B is 
executed to descramble the symbob, thereby obtaining a 
matrix conesponding to die symbol matrix 500 widi the 

25 error checking codes before scrambling (14-5). This 
descrambling processing stq> 14-5 can also be understood 
from the description about the scrambling processing, and a 
detailed description thereof will be omitted. 
A difference between the descrambled symbol matrix with 

so the error checking codes and the symbol matrix with the 
error checking codes which is formed in encoding is an 
error. Thus, in step 14-6, ecror correction is executed using 
the error checking codes, thus obtaining target data. 
Of the series of these processing operations, a matrix, in 

35 Which black and white levels of meshes are indicated by bits, 
corresponding to the last two-dimensional bit matrix 1100 is 
obtained as a result of sampling based on &e scanning 
reference patterns. Under a user environment wherein a 
scanning spetd and a scanning direction can be kept con- 

40 stant using a mechanical scanning type image sensor, even 
if no scanning reference patterns are arranged, meshes can 
be dirccdy recognized bom im^e data which is free fi^om 
image distortion, as has been described in the second 
embodiment. 

45 The error correction step 14-6 as the characteristic feature 
of tfiis embodiment will be described in detail below. FIG. 
36 shows the entire error ccaxection flow. Before this flow 
starts, a descrambled two-dimensional symbol matrix 500R 
with error checking codes is obtained as shown in RG. 35. 

50 PO to P7 in FIG. 35 respectively indicate first to eighth lines 
of die two-dimensional matrix 500R. Since error checking 
codes £ are added to these lines, these lines are called P 
vectors in FIG. 36. Since error checking codes g are added 
to columns (indicated by QO to Q35), of the two- 

55 dimensional matrix, these columns are called Q vectors in 
FIG 36. In the loop including stq>$ 24-1 to 24-3 in FIG. 36, 
error correction processing step 24-2 is executed once for 
the 36 Q vectors. Thereafter, in the loop including steps 24-4 
to 24-6. error cociection processing step 24-5 is executed 

60 once for the eight F vectors. Thereafter, the flow advances to 
stq> 24-7 to check if correction is actually executed in steps 
24-2 

and 24-5 or erasure registratton necessary for subsequent 
correction Is petf ormed although no correction is executed. 
65 Since it can be considered that an error remains excluding a 
case wherein neitho' of these operations are performed, a 
correction count is reset to zero, and steps 24-1 to 24-5 are 
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executed under a condition that the number of Loop times is 
small (5 or less in FIG. 36). When the number of Loop times 
is large, considerable errors are included in the matrix 500R. 
and even if error correction is continued, error correction 
must be further repeated. Thus, the processing is stopped in 5 
this case. 

FIG. 37 shows error concction processing steps 24-2 and 
24-5 in detail. In step 2S-1. calculations corresponding to the 
left-hand side of equation (7) arc executed for the Q vcctcffs, 
and calculations corresponding to the left-hand side of lO 
equation (8) arc executed for the P vectors, thereby obtaining 
syndrome values. That is, a syndrome matrix is multiplied 
with a vector of interest If there is no error in tiie vector 
(block) of interest syndrome values (Sq, S^, S^, S3) become 
zero, as shown in equations (7) and (8). Thus, the syndrome 15 
values are checked in step 25-2. If all the syndrome values 
are zero»thc flow returns to the main routine since no eiros' 
is detected. Thus, an operation of checking a no-error state 
which has the highest possibility is completed. An error 
having the second highest possibility is a single-symbol 20 
error, and an eiror having the third highest possibility is 
double- symbol errors. Since four error checking codes arc 
added per vector, correction can be made up to the double 
errors. When the syndrome values become non-zero, an 
error of one or two symbols is assumed. If two symbols in 25 
one vector include errors, location values representing circa* 
positions are represented by Xi and X3, and error patterns 
are presented by and Yj, the following relations are 
established for the syndrome values So» S^, Sj* and S3 which 
have already been calculated in step 25-1: 30 

Thus, four siimdtaneous equations foe four unlmowns X^, 
Xj, Yp and Y2 are obtained. The location values Xi and Xj 40 
become a root of AXVBX-+O0 for: 

43 

In step 25-3. values of A, B, and C are calculated. In 
consideration of a possibility of a single-symbol error, since 
Yj or Y2 is zero, A=0 is established. Thus, it is checked in ^ 
step 25-4 if A=0. If A=0, a single-symbol error is 
determined, and its error pattern (Y=So) and a location value 
(X=Si/So) are obtained. The error pattern is added 
(EXORed) to a symbol specified by the location value, thus 
correcting the single-symbol error. Thereafter, a correction 
count is inaemented by one (25-5 to 25-7). 

If A=0 is not established, D (=B/A) and E (=C/A) arc 
calculated (25-8), and the following quadratic expression is 
solved (25-9): 

(X/DfHX/oy^ (10) ^ 

If double-symbol enors occur, two real roots are obtained. If 
the two real roots are obtained io step 25-10. double-symbol 
errors are determined, and error location values X| and X2 
(X| is D times of a solution R of equatioo ( 10), i.e.. DR, and 65 
X2 is given by X2=D+Xi), and error patterns Y^ and Yj 
(Y2=KXrSo+Si)/CXi+X2), Ya=So+Y2) are calculated. The 



30 

error patterns arc respectively added to the two symbols 
indicated by the two error location values to correct these 
symbols, and the correction count is incremented by two 
(25-11 to 25-13). If NO is determined in step 25-10, since 
three oc more oiors occur and cannot be corrected by this 
correction alone, the flow advances to erasure jrocessing 
step 25-14 utilizing results of vectors in other systems. 

FIG. 38 shows an example of erasure processing step 
25-14. In step 26-1, erasure-registered positions of a vector 
of interest are counted. For example, if vectors to Pj are 
erasure-registered for the vector (Jo in FIG. 35, the positions 
of the symbols Sq, S^. and S2 have a high possibility of an 
error. It is checked in step 26-2 if the number of erasure- 
registered position is 3. If tiiere is no erroneous correction so 
far, symbols at positions which are not erasure-registered 
must be correct Therefore, if triple-symbol errors occur, 
their positions must be erasure-registered. Thus, if YES is 
determined in step 26-2, triple-symbol errors are 
determined, and error patterns are calculated using the three 
erasure-registered positions as error locations. For example, 
if the three error location values are represented by Xj, X^, 
and X3, and their error patterns are represented by Yj. Y^, 
and Y3, the following equations are established: 

Si=XJ,+X2Y2+X3Y, 

Si=x^^yi+X2^yj+X3% 

Since only the error patterns Yj. Y^, and Y3 are unknown in 
these three simultaneous equations, their solutions can be 
obtained. For cxamfdc. Yj can be calculated by: 

„ XiXiSo + iXi + Xi^Si-i-Si 

In step 26-5, enror pattans are re^)ectiveLy added to three 
symbols on the matrix associated with errors to correct these 
symbols, and the correction count is incremented by 3 in 
step 26-6. 

If the number of erasure-registered positions is not 3 in 
step 26-2, the position of the vector of interest is erasure- 
registered to give an opportunity of erasure processing by 
vectors of another system (Q vector) different from the 
vectOT of interest (e.g., P vector) (26-7). 

FIGS. 39{ayd9(g) show examples of burst errors on the 
encoded image 20. In FIGS. 39(a) to 39(^), contiguous black 
areas indicate burst errors. According to this embodiment, 
target data could be correctly obtained even if any burst 
errors shown in HGS. 39(a) to 39{g) occur. 

Embodiments of the present invention have been 
described. Various other changes and modifications may be 
made within the spirit and scope of the invention. For 
exaii9>le, the data recording ai^aratus and the data r^ro- 
ducing apparatus can be easily integrated to constitute a 
recording/reproduction unit The mesh pattern 22 as a two- 
dimensional mesh matrix without any gap is used as a data 
body of the encoded inoage 20 but may be a two-dimensional 
matrix of image elements including gaps. A photoelectric 
conversion type image sensor is employed in the above 
embodiments. However, a magnetic type image sensor 
which responds to a magnetic ink or the like may be used. 
In the above embodiments, scrambling (interleaving) pro- 
cessing and randomization processing of element values arc 
separately executed. For exanq)lc, position saambling for 
moving element positions (i j) on a two-dimensional matrix 
to other positions using a random number table may be 
executed to simultaneously remove DC con^nents. 
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Acoordiiig to one arrangctnent of the prescat invcntioiL 
when an encoded image on a recoiding medium is coBsti- 
tuted by a two-dimensional matrix of image dements 
expressing data elements, cnoi checking codes are added to 
a two-dimensional data matrix constituting taiget data as 5 
original information, and the encoded image as the two- 
dimensional matrix of image elements is printed on the 
recording medium according to the two-dimensional data 
matrix with the error checking codes. Therefore, a recording 
medium which is not easily influenced by errors can be lO 
provided even at a relatively high recording density. In 
addition, printing precision can be reduced according to 
error correction capability of error checking codes. 

Furthermore, according to one anangement of the present 
invention . data of an encoded image is read from a recording 15 
mediuno. image elements expressing data elements are 
detected from the read image data to obtain a two- 
dimensional data matrix, and an error on the matrix is 
corrected according to error checking codes included in this 
matrix, thereby reproducing target data. Thus* target data 20 
can be correctly reproduced even if an error occurs on a 
two-dimensional data matrix when an encoded image is 
read, when image elements are recognized from the image 
data, or due to error factors before these processes. 

Since a means for changing die order of a data matrix with 25 
error checking codes in recording and for reversing the order 
to rest<H-e the original data matrix with the enor checking 
codes in reproducticHi is arranged, characteristics strong 
against burst eirors can be obtained. More specifically, even 
when errors continuously occur in recogoition of image 30 
elements during reproduction due to a considerably large 
contaminatioa on an encoded image on a recording medium, 
the burst errors can be converted to discrete cnt>rs on the 
original data matrix with the error checking codes. 
Therefore, these errors can be easily corrected within the 35 
error correction capability of the error checking codes. As a 
result, reproducibility of correct target data can be improved. 

Furthermore, according to one arrangement of the present 
invention, a read error of image elements of an encoded 
image caused by a change in conversion characteristics of an 40 
image sensor, espedally. a change in characteristics caused 
by DC components, i.e., continuation of the same density is 
taken into consideration. When an encoded image is 
recorded, element values of a data matrix constituting given 
data are randomized using pseudo random numbers. Upon as 
rejproductioD. after the randomized data matrix is obtained, 
(he data matrix is derandomized to restore element values of 
the original data matrix. Therefore, no long mesh arrays 
having ttie same density appear in the encoded image on the 
recording medium. When the encoded image is read, pixels so 
can be accurately digitized (e.g,^ binarized). and an euor in 
reproduced data caused by a read error can be prevented. 

Some embodiments of the present invention have been 
described. However, these embodiments are merely 
cxanqiles, and the present invention may have various other 55 
arrangements. All the modifications and applications of the 
present invcndon are incorporated in die scope of the 
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invention, and, hence, the scope of die present invention 
should be determined by only the ^>pendcd claims and their 
equivalents. 
What is claimed is: 

1. A data reproduction method comprising the steps of: 
reading an encoded image as image data from a recording 

medium on which image elements including a mesh 
pattern having a rectangular shape as a whole and 
including dark and light image elements wliidi are 
arranged at two-dimensional distributed positions, 
whose arrangement is determined based on a changed 
two-dimensional matrix obtained by changing an (^igi- 
nal two-dimensional matrix constituting target data in 
accordance with predetermined pseudo random 
numbers, respective randomized target data b„ being 
obtained by executing an exclusive OR operation for 
target data b„ with pseudo random numbers md(n); 
detecting image elements from the read image data, and 
obtaining the elements of the randomized data to form 
the changed two-dimensional matrix; and 
changing die elements on the formed two-dimensional 
matrix according to the predetenninod pseudo random 
numbers to form the original two-dimensional matrix, 
thereby reproducing the target data, said target data b„ 
being obtained by executing an exclusive OR operation 
for the randomized target data b„ with die same ps^do 
random numbers rnd(n). 

2. A data reproduction apparatus conqnising: 
image sensor means for reading an encoded image as 

image data from a recording medium on which image 
elements including a mesh pattern having a rectangular 
sh^ as a whole and including dark and light image 
elements which arc arranged at two-dimensional dis- 
tributed positions, and whose arrangement is deter- 
mined based on a changed two-dimensional matrix 
obtained by changing an original two-dimensional 
matrix constituting target data in accordance with pre- 
determined pseudo random numbers, respective ran- 
domized target data b„ being obtained by executing an 
exclusive OR operation for target data b„ with pseudo 
random numbers md(n); 
image element decoding means for detecting image de- 
ments from the read image data, and obtaining the 
elements from the read image data, and obtaining die 
elements of the randomized data to form the changed 
two-dimensional matrix; and 
derandomizing means for changing the elements on the 
formed two-dimensional matrix according to the pre- 
determined pseudo random numbers to form the origi- 
nal two-dimensional matrix, thereby reproducing the 
target data, said target data b„ being obtained by 
executing an exclusive OR operation for the random- 
ized target data b^ with the same pseudo random 
numbers md (n). 

***** 
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